Phased DMD: Few-step Distribution Matching Distillation via Score
  Matching within Subintervals
Distribution Matching Distillation (DMD) distills score-based generative
models into efficient one-step generators, without requiring a one-to-one
correspondence with the sampling trajectories of their teachers. However,
limited model capacity causes one-step distilled models underperform on complex
generative tasks, e.g., synthesizing intricate object motions in text-to-video
generation. Directly extending DMD to multi-step distillation increases memory
usage and computational depth, leading to instability and reduced efficiency.
While prior works propose stochastic gradient truncation as a potential
solution, we observe that it substantially reduces the generation diversity of
multi-step distilled models, bringing it down to the level of their one-step
counterparts. To address these limitations, we propose Phased DMD, a multi-step
distillation framework that bridges the idea of phase-wise distillation with
Mixture-of-Experts (MoE), reducing learning difficulty while enhancing model
capacity. Phased DMD is built upon two key ideas: progressive distribution
matching and score matching within subintervals. First, our model divides the
SNR range into subintervals, progressively refining the model to higher SNR
levels, to better capture complex distributions. Next, to ensure the training
objective within each subinterval is accurate, we have conducted rigorous
mathematical derivations. We validate Phased DMD by distilling state-of-the-art
image and video generation models, including Qwen-Image (20B parameters) and
Wan2.2 (28B parameters). Experimental results demonstrate that Phased DMD
preserves output diversity better than DMD while retaining key generative
capabilities. We will release our code and models.
690817db812eca10f9cc5e44
Xiangyu Fan
690817db812eca10f9cc5e45
Zesong Qiu
690817db812eca10f9cc5e46
Zhuguanyu Wu
690817db812eca10f9cc5e47
Fanzhou Wang
690817db812eca10f9cc5e48
Zhiqian Lin
690817db812eca10f9cc5e49
Tianxiang Ren
690817db812eca10f9cc5e4a
Dahua Lin
690817db812eca10f9cc5e4b
Ruihao Gong
690817db812eca10f9cc5e4c
Lei Yang
claimed_verified
2025-11-10T09:32:45.615Z
6626a471430a124253f197c8
https://cdn-avatars.huggingface.co/v1/production/uploads/6626a471430a124253f197c8/uVEk5nnW-bS6-no0rQ7Wh.png
yl-1993
yl-1993
user
yl-1993
Phased DMD: Few-step Distribution Matching Distillation via Score
  Matching within Subintervals
Distribution Matching Distillation (DMD) distills score-based generative
models into efficient one-step generators, without requiring a one-to-one
correspondence with the sampling trajectories of their teachers. However,
limited model capacity causes one-step distilled models underperform on complex
generative tasks, e.g., synthesizing intricate object motions in text-to-video
generation. Directly extending DMD to multi-step distillation increases memory
usage and computational depth, leading to instability and reduced efficiency.
While prior works propose stochastic gradient truncation as a potential
solution, we observe that it substantially reduces the generation diversity of
multi-step distilled models, bringing it down to the level of their one-step
counterparts. To address these limitations, we propose Phased DMD, a multi-step
distillation framework that bridges the idea of phase-wise distillation with
Mixture-of-Experts (MoE), reducing learning difficulty while enhancing model
capacity. Phased DMD is built upon two key ideas: progressive distribution
matching and score matching within subintervals. First, our model divides the
SNR range into subintervals, progressively refining the model to higher SNR
levels, to better capture complex distributions. Next, to ensure the training
objective within each subinterval is accurate, we have conducted rigorous
mathematical derivations. We validate Phased DMD by distilling state-of-the-art
image and video generation models, including Qwen-Image (20B parameters) and
Wan2.2 (28B parameters). Experimental results demonstrate that Phased DMD
preserves output diversity better than DMD while retaining key generative
capabilities. We will release our code and models.
Phased DMD enhances multi-step distillation of score-based generative models by dividing SNR ranges and using progressive distribution matching, improving diversity and generative capabilities.
score-based generative models distribution matching distillation one-step generators multi-step distillation memory usage computational depth stochastic gradient truncation generation diversity phase-wise distillation Mixture-of-Experts progressive distribution matching score matching subintervals SNR range Qwen-Image Wan2.2