Accelerating Masked Image Generation by Learning Latent Controlled Dynamics
Masked Image Generation Models (MIGMs) have achieved great success, yet their efficiency is hampered by the multiple steps of bi-directional attention. In fact, there exists notable redundancy in their computation: when sampling discrete tokens, the rich semantics contained in the continuous features are lost. Some existing works attempt to cache the features to approximate future features. However, they exhibit considerable approximation error under aggressive acceleration rates. We attribute this to their limited expressivity and the failure to account for sampling information. To fill this gap, we propose to learn a lightweight model that incorporates both previous features and sampled tokens, and regresses the average velocity field of feature evolution. The model has moderate complexity that suffices to capture the subtle dynamics while keeping lightweight compared to the original base model. We apply our method, MIGM-Shortcut, to two representative MIGM architectures and tasks. In particular, on the state-of-the-art Lumina-DiMOO, it achieves over 4x acceleration of text-to-image generation while maintaining quality, significantly pushing the Pareto frontier of masked image generation. The code and model weights are available at https://github.com/Kaiwen-Zhu/MIGM-Shortcut.
69a4fbffbc2d66e68888f4e4
Kaiwen Zhu
claimed_verified
2026-03-02T09:37:37.446Z
63d2775d5c52bbd72cac2e86
/avatars/88a1391352be2c760ee033fe456dd738.svg
KaiwenZhu
Kaiwen-Zhu
user
Kaiwen-Zhu
69a4fbffbc2d66e68888f4e5
Quansheng Zeng
claimed_verified
2026-03-02T10:56:26.617Z
640f078206c3b5ca883ea522
/avatars/e1707f12ae833a956c264008bddf8508.svg
Quan-Sheng Zeng (SII)
ashun989
user
ashun989
69a4fbffbc2d66e68888f4e6
Yuandong Pu
69a4fbffbc2d66e68888f4e7
Shuo Cao
claimed_verified
2026-03-02T09:37:35.498Z
676a7d235303833e60a3edc5
https://cdn-avatars.huggingface.co/v1/production/uploads/676a7d235303833e60a3edc5/-hmwUmlm-IYvwpH8P5AuA.jpeg
caoshuo
Thunderbolt215215
user
Thunderbolt215215
69a4fbffbc2d66e68888f4e8
Xiaohui Li
69a4fbffbc2d66e68888f4e9
Yi Xin
69a4fbffbc2d66e68888f4ea
Qi Qin
69a4fbffbc2d66e68888f4eb
Jiayang Li
69a4fbffbc2d66e68888f4ec
Yu Qiao
69a4fbffbc2d66e68888f4ed
Jinjin Gu
69a4fbffbc2d66e68888f4ee
Yihao Liu
Accelerating Masked Image Generation by Learning Latent Controlled Dynamics
Masked Image Generation Models (MIGMs) have achieved great success, yet their efficiency is hampered by the multiple steps of bi-directional attention. In fact, there exists notable redundancy in their computation: when sampling discrete tokens, the rich semantics contained in the continuous features are lost. Some existing works attempt to cache the features to approximate future features. However, they exhibit considerable approximation error under aggressive acceleration rates. We attribute this to their limited expressivity and the failure to account for sampling information. To fill this gap, we propose to learn a lightweight model that incorporates both previous features and sampled tokens, and regresses the average velocity field of feature evolution. The model has moderate complexity that suffices to capture the subtle dynamics while keeping lightweight compared to the original base model. We apply our method, MIGM-Shortcut, to two representative MIGM architectures and tasks. In particular, on the state-of-the-art Lumina-DiMOO, it achieves over 4x acceleration of text-to-image generation while maintaining quality, significantly pushing the Pareto frontier of masked image generation. The code and model weights are available at https://github.com/Kaiwen-Zhu/MIGM-Shortcut.
MIGM-Shortcut accelerates masked image generation by learning a lightweight model that predicts feature evolution velocity, achieving over 4x speedup with maintained quality.
masked image generation models bi-directional attention discrete tokens continuous features feature evolution velocity field regression lightweight model text-to-image generation Lumina-DiMOO Pareto frontier