Distilled Decoding 2: One-step Sampling of Image Auto-regressive Models
  with Conditional Score Distillation
Image Auto-regressive (AR) models have emerged as a powerful paradigm of
visual generative models. Despite their promising performance, they suffer from
slow generation speed due to the large number of sampling steps required.
Although Distilled Decoding 1 (DD1) was recently proposed to enable few-step
sampling for image AR models, it still incurs significant performance
degradation in the one-step setting, and relies on a pre-defined mapping that
limits its flexibility. In this work, we propose a new method, Distilled
Decoding 2 (DD2), to further advances the feasibility of one-step sampling for
image AR models. Unlike DD1, DD2 does not without rely on a pre-defined
mapping. We view the original AR model as a teacher model which provides the
ground truth conditional score in the latent embedding space at each token
position. Based on this, we propose a novel conditional score
distillation loss to train a one-step generator. Specifically, we train a
separate network to predict the conditional score of the generated distribution
and apply score distillation at every token position conditioned on previous
tokens. Experimental results show that DD2 enables one-step sampling for image
AR models with an minimal FID increase from 3.40 to 5.43 on ImageNet-256.
Compared to the strongest baseline DD1, DD2 reduces the gap between the
one-step sampling and original AR model by 67%, with up to 12.3times
training speed-up simultaneously. DD2 takes a significant step toward the goal
of one-step AR generation, opening up new possibilities for fast and
high-quality AR modeling. Code is available at
https://github.com/imagination-research/Distilled-Decoding-2.
68fff75322d452aac6dd4260
Enshu Liu
claimed_verified
2025-10-28T15:36:51.770Z
63c7ecbcaa2a7669c45ad49e
/avatars/eba0250d328b0d850d3a6d3057bea583.svg
Enshu Liu
jsttlgdkycy
user
jsttlgdkycy
68fff75322d452aac6dd4261
Qian Chen
68fff75322d452aac6dd4262
Xuefei Ning
68fff75322d452aac6dd4263
Shengen Yan
68fff75322d452aac6dd4264
Guohao Dai
68fff75322d452aac6dd4265
Zinan Lin
claimed_verified
2025-10-28T15:36:53.977Z
64c832a8c547ed5243d29630
https://cdn-avatars.huggingface.co/v1/production/uploads/64c832a8c547ed5243d29630/a46V0xOVRVknM9mUjKkTf.jpeg
Zinan Lin
fjxmlzn
user
fjxmlzn
68fff75322d452aac6dd4266
Yu Wang
Distilled Decoding 2: One-step Sampling of Image Auto-regressive Models
  with Conditional Score Distillation
Image Auto-regressive (AR) models have emerged as a powerful paradigm of
visual generative models. Despite their promising performance, they suffer from
slow generation speed due to the large number of sampling steps required.
Although Distilled Decoding 1 (DD1) was recently proposed to enable few-step
sampling for image AR models, it still incurs significant performance
degradation in the one-step setting, and relies on a pre-defined mapping that
limits its flexibility. In this work, we propose a new method, Distilled
Decoding 2 (DD2), to further advances the feasibility of one-step sampling for
image AR models. Unlike DD1, DD2 does not without rely on a pre-defined
mapping. We view the original AR model as a teacher model which provides the
ground truth conditional score in the latent embedding space at each token
position. Based on this, we propose a novel conditional score
distillation loss to train a one-step generator. Specifically, we train a
separate network to predict the conditional score of the generated distribution
and apply score distillation at every token position conditioned on previous
tokens. Experimental results show that DD2 enables one-step sampling for image
AR models with an minimal FID increase from 3.40 to 5.43 on ImageNet-256.
Compared to the strongest baseline DD1, DD2 reduces the gap between the
one-step sampling and original AR model by 67%, with up to 12.3times
training speed-up simultaneously. DD2 takes a significant step toward the goal
of one-step AR generation, opening up new possibilities for fast and
high-quality AR modeling. Code is available at
https://github.com/imagination-research/Distilled-Decoding-2.
A new method, Distilled Decoding 2 (DD2), enables one-step sampling for image auto-regressive models with minimal performance degradation and significant speed-up compared to previous methods.
image auto-regressive models Distilled Decoding 1 (DD1) Distilled Decoding 2 (DD2) conditional score distillation loss latent embedding space token position one-step sampling FID ImageNet-256