Decoupled DMD: CFG Augmentation as the Spear, Distribution Matching as the Shield
Diffusion model distillation has emerged as a powerful technique for creating efficient few-step and single-step generators. Among these, Distribution Matching Distillation (DMD) and its variants stand out for their impressive performance, which is widely attributed to their core mechanism of matching the student's output distribution to that of a pre-trained teacher model. In this work, we challenge this conventional understanding. Through a rigorous decomposition of the DMD training objective, we reveal that in complex tasks like text-to-image generation, where CFG is typically required for desirable few-step performance, the primary driver of few-step distillation is not distribution matching, but a previously overlooked component we identify as CFG Augmentation (CA). We demonstrate that this term acts as the core ``engine'' of distillation, while the Distribution Matching (DM) term functions as a ``regularizer'' that ensures training stability and mitigates artifacts. We further validate this decoupling by demonstrating that while the DM term is a highly effective regularizer, it is not unique; simpler non-parametric constraints or GAN-based objectives can serve the same stabilizing function, albeit with different trade-offs. This decoupling of labor motivates a more principled analysis of the properties of both terms, leading to a more systematic and in-depth understanding. This new understanding further enables us to propose principled modifications to the distillation process, such as decoupling the noise schedules for the engine and the regularizer, leading to further performance gains. Notably, our method has been adopted by the Z-Image ( https://github.com/Tongyi-MAI/Z-Image ) project to develop a top-tier 8-step image generation model, empirically validating the generalization and robustness of our findings.
692d12574397b1ec214f6850
Dongyang Liu
claimed_verified
2025-12-01T16:24:40.998Z
646f1bef075e11ca78da3bb7
https://cdn-avatars.huggingface.co/v1/production/uploads/646f1bef075e11ca78da3bb7/gNS-ikyZXYeMrf4a7HTQE.jpeg
Dongyang Liu (Chris Liu)
Cxxs
user
Cxxs
692d12574397b1ec214f6851
Peng Gao
692d12574397b1ec214f6852
David Liu
692d12574397b1ec214f6853
Ruoyi Du
claimed_verified
2025-12-03T09:18:50.255Z
64a54586c0f13de8e7093314
/avatars/389e43e9a32cf2fc95f8f3a23b8f0508.svg
Ruoyi Du
RuoyiDu
user
RuoyiDu
692d12574397b1ec214f6854
Zhen Li
claimed_verified
2025-12-01T09:10:43.914Z
6285a9133ab6642179158944
/avatars/6e10fa07c94141fcdbe0cab02bb731ca.svg
Zhen Li
Paper99
user
Paper99
692d12574397b1ec214f6855
Qilong Wu
claimed_verified
2025-12-01T09:10:41.744Z
64379d79fac5ea753f1c10f3
https://cdn-avatars.huggingface.co/v1/production/uploads/64379d79fac5ea753f1c10f3/clfjIaMTVDTG9K04dRud_.png
Jerry Wu
QJerry
user
QJerry
692d12574397b1ec214f6856
Xin Jin
claimed_verified
2025-12-01T09:10:39.946Z
6537e8eab01250d1d6efed3a
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/gMx73gwdfEhcCFioStGCE.jpeg
Xin
Srameo
user
Srameo
692d12574397b1ec214f6857
Sihan Cao
692d12574397b1ec214f6858
Shifeng Zhang
692d12574397b1ec214f6859
Hongsheng Li
692d12574397b1ec214f685a
Steven Hoi
Decoupled DMD: CFG Augmentation as the Spear, Distribution Matching as the Shield
Diffusion model distillation has emerged as a powerful technique for creating efficient few-step and single-step generators. Among these, Distribution Matching Distillation (DMD) and its variants stand out for their impressive performance, which is widely attributed to their core mechanism of matching the student's output distribution to that of a pre-trained teacher model. In this work, we challenge this conventional understanding. Through a rigorous decomposition of the DMD training objective, we reveal that in complex tasks like text-to-image generation, where CFG is typically required for desirable few-step performance, the primary driver of few-step distillation is not distribution matching, but a previously overlooked component we identify as CFG Augmentation (CA). We demonstrate that this term acts as the core ``engine'' of distillation, while the Distribution Matching (DM) term functions as a ``regularizer'' that ensures training stability and mitigates artifacts. We further validate this decoupling by demonstrating that while the DM term is a highly effective regularizer, it is not unique; simpler non-parametric constraints or GAN-based objectives can serve the same stabilizing function, albeit with different trade-offs. This decoupling of labor motivates a more principled analysis of the properties of both terms, leading to a more systematic and in-depth understanding. This new understanding further enables us to propose principled modifications to the distillation process, such as decoupling the noise schedules for the engine and the regularizer, leading to further performance gains. Notably, our method has been adopted by the Z-Image ( https://github.com/Tongyi-MAI/Z-Image ) project to develop a top-tier 8-step image generation model, empirically validating the generalization and robustness of our findings.
The study reveals that in text-to-image generation, CFG Augmentation is the primary driver of few-step distillation in Distribution Matching Distillation (DMD), while the distribution matching term acts as a regularizer.
diffusion model distillation Distribution Matching Distillation (DMD) CFG Augmentation (CA) text-to-image generation configuration (CFG) distribution matching noise schedules Z-Image