RecTok: Reconstruction Distillation along Rectified Flow
Visual tokenizers play a crucial role in diffusion models. The dimensionality of latent space governs both reconstruction fidelity and the semantic expressiveness of the latent feature. However, a fundamental trade-off is inherent between dimensionality and generation quality, constraining existing methods to low-dimensional latent spaces. Although recent works have leveraged vision foundation models to enrich the semantics of visual tokenizers and accelerate convergence, high-dimensional tokenizers still underperform their low-dimensional counterparts. In this work, we propose RecTok, which overcomes the limitations of high-dimensional visual tokenizers through two key innovations: flow semantic distillation and reconstruction--alignment distillation. Our key insight is to make the forward flow in flow matching semantically rich, which serves as the training space of diffusion transformers, rather than focusing on the latent space as in previous works. Specifically, our method distills the semantic information in VFMs into the forward flow trajectories in flow matching. And we further enhance the semantics by introducing a masked feature reconstruction loss. Our RecTok achieves superior image reconstruction, generation quality, and discriminative performance. It achieves state-of-the-art results on the gFID-50K under both with and without classifier-free guidance settings, while maintaining a semantically rich latent space structure. Furthermore, as the latent dimensionality increases, we observe consistent improvements. Code and model are available at https://shi-qingyu.github.io/rectok.github.io.
6940fd0b65f1e24a117806c0
Qingyu Shi
claimed_verified
2025-12-16T09:37:12.941Z
656724074f6ec72017754d33
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/zLIgt4xvfbYPLZh0E3WWF.png
QingyuShi
QingyuShi
user
QingyuShi
6940fd0b65f1e24a117806c1
Size Wu
6940fd0b65f1e24a117806c2
Jinbin Bai
claimed_verified
2025-12-16T09:37:10.480Z
63fccdac93b993a4ebd7789a
https://cdn-avatars.huggingface.co/v1/production/uploads/63fccdac93b993a4ebd7789a/KRx8vpdoDjsZBRw0j8Vg8.jpeg
Jinbin Bai
BryanW
user
BryanW
6940fd0b65f1e24a117806c3
Kaidong Yu
6940fd0b65f1e24a117806c4
Yujing Wang
6940fd0b65f1e24a117806c5
Yunhai Tong
6940fd0b65f1e24a117806c6
Xiangtai Li
6940fd0b65f1e24a117806c7
Xuelong Li
RecTok: Reconstruction Distillation along Rectified Flow
Visual tokenizers play a crucial role in diffusion models. The dimensionality of latent space governs both reconstruction fidelity and the semantic expressiveness of the latent feature. However, a fundamental trade-off is inherent between dimensionality and generation quality, constraining existing methods to low-dimensional latent spaces. Although recent works have leveraged vision foundation models to enrich the semantics of visual tokenizers and accelerate convergence, high-dimensional tokenizers still underperform their low-dimensional counterparts. In this work, we propose RecTok, which overcomes the limitations of high-dimensional visual tokenizers through two key innovations: flow semantic distillation and reconstruction--alignment distillation. Our key insight is to make the forward flow in flow matching semantically rich, which serves as the training space of diffusion transformers, rather than focusing on the latent space as in previous works. Specifically, our method distills the semantic information in VFMs into the forward flow trajectories in flow matching. And we further enhance the semantics by introducing a masked feature reconstruction loss. Our RecTok achieves superior image reconstruction, generation quality, and discriminative performance. It achieves state-of-the-art results on the gFID-50K under both with and without classifier-free guidance settings, while maintaining a semantically rich latent space structure. Furthermore, as the latent dimensionality increases, we observe consistent improvements. Code and model are available at https://shi-qingyu.github.io/rectok.github.io.
RecTok improves diffusion models by enriching forward flow semantics and enhancing reconstruction, achieving state-of-the-art results with high-dimensional visual tokenizers.
visual tokenizers diffusion models latent space dimensionality generation quality vision foundation models RecTok flow semantic distillation reconstruction--alignment distillation flow matching diffusion transformers masked feature reconstruction loss gFID-50K classifier-free guidance