Gated Condition Injection without Multimodal Attention: Towards Controllable Linear-Attention Transformers
Recent advances in diffusion-based controllable visual generation have led to remarkable improvements in image quality. However, these powerful models are typically deployed on cloud servers due to their large computational demands, raising serious concerns about user data privacy. To enable secure and efficient on-device generation, we explore in this paper controllable diffusion models built upon linear attention architectures, which offer superior scalability and efficiency, even on edge devices. Yet, our experiments reveal that existing controllable generation frameworks, such as ControlNet and OminiControl, either lack the flexibility to support multiple heterogeneous condition types or suffer from slow convergence on such linear-attention models. To address these limitations, we propose a novel controllable diffusion framework tailored for linear attention backbones like SANA. The core of our method lies in a unified gated conditioning module working in a dual-path pipeline, which effectively integrates multi-type conditional inputs, such as spatially aligned and non-aligned cues. Extensive experiments on multiple tasks and benchmarks demonstrate that our approach achieves state-of-the-art controllable generation performance based on linear-attention models, surpassing existing methods in terms of fidelity and controllability.
69cf3a53e02fdd68b91b7a9d
Yuhe Liu
claimed_verified
2026-04-07T08:45:33.588Z
6761a439b4b56a823c8bce1b
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/fUKU8eOtdru96Lfy5gFAa.png
liu
linlinlinlinsdada
user
linlinlinlinsdada
69cf3a53e02fdd68b91b7a9e
Zhenxiong Tan
claimed_verified
2026-04-03T09:07:42.183Z
640ebdfefdeaae139086f4d8
https://cdn-avatars.huggingface.co/v1/production/uploads/640ebdfefdeaae139086f4d8/2N94gbHubplYD8njmUTPf.jpeg
Zhenxiong Tan
Yuanshi
user
Yuanshi
69cf3a53e02fdd68b91b7a9f
Yujia Hu
69cf3a53e02fdd68b91b7aa0
Songhua Liu
69cf3a53e02fdd68b91b7aa1
Xinchao Wang
Gated Condition Injection without Multimodal Attention: Towards Controllable Linear-Attention Transformers
Recent advances in diffusion-based controllable visual generation have led to remarkable improvements in image quality. However, these powerful models are typically deployed on cloud servers due to their large computational demands, raising serious concerns about user data privacy. To enable secure and efficient on-device generation, we explore in this paper controllable diffusion models built upon linear attention architectures, which offer superior scalability and efficiency, even on edge devices. Yet, our experiments reveal that existing controllable generation frameworks, such as ControlNet and OminiControl, either lack the flexibility to support multiple heterogeneous condition types or suffer from slow convergence on such linear-attention models. To address these limitations, we propose a novel controllable diffusion framework tailored for linear attention backbones like SANA. The core of our method lies in a unified gated conditioning module working in a dual-path pipeline, which effectively integrates multi-type conditional inputs, such as spatially aligned and non-aligned cues. Extensive experiments on multiple tasks and benchmarks demonstrate that our approach achieves state-of-the-art controllable generation performance based on linear-attention models, surpassing existing methods in terms of fidelity and controllability.
Controllable diffusion models using linear attention architectures enable secure on-device visual generation with improved multi-condition input handling and faster convergence.
diffusion models linear attention controllable visual generation ControlNet OminiControl SANA gated conditioning module dual-path pipeline spatially aligned cues non-aligned cues