TC-AE: Unlocking Token Capacity for Deep Compression Autoencoders
We propose TC-AE, a ViT-based architecture for deep compression autoencoders. Existing methods commonly increase the channel number of latent representations to maintain reconstruction quality under high compression ratios. However, this strategy often leads to latent representation collapse, which degrades generative performance. Instead of relying on increasingly complex architectures or multi-stage training schemes, TC-AE addresses this challenge from the perspective of the token space, the key bridge between pixels and image latents, through two complementary innovations: Firstly, we study token number scaling by adjusting the patch size in ViT under a fixed latent budget, and identify aggressive token-to-latent compression as the key factor that limits effective scaling. To address this issue, we decompose token-to-latent compression into two stages, reducing structural information loss and enabling effective token number scaling for generation. Secondly, to further mitigate latent representation collapse, we enhance the semantic structure of image tokens via joint self-supervised training, leading to more generative-friendly latents. With these designs, TC-AE achieves substantially improved reconstruction and generative performance under deep compression. We hope our research will advance ViT-based tokenizer for visual generation.
69d717fbc6ed9968ac5c699c
Teng Li
admin_assigned
2026-04-09T08:16:10.394Z
677a3eb68b252e571a1fce1a
/avatars/f2260ef6ade5cb22606516f26e872b57.svg
Teng LI
tliby
user
tliby
69d717fbc6ed9968ac5c699d
Ziyuan Huang
claimed_verified
2026-04-09T18:50:48.378Z
655c5709948930b0fcf88a22
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/4Qk9Nlyus-su16GdIPJao.png
Ziyuan
zyhuangnus
user
zyhuangnus
69d717fbc6ed9968ac5c699e
Cong Chen
69d717fbc6ed9968ac5c699f
Yangfu Li
admin_assigned
2026-04-09T08:17:13.357Z
63578d0e265b76ad9ede930f
/avatars/5457ff2508686c7dd60288fb232b3179.svg
liyangfu
lyf971215
user
lyf971215
69d717fbc6ed9968ac5c69a0
Yuanhuiyi Lyu
admin_assigned
2026-04-09T08:17:20.316Z
64e019e5f025db94263ca91b
/avatars/a73a962c47eb548bf55dab9f2b5c3883.svg
YuanhuiyiLyu
YuanhuiyiLyu
user
YuanhuiyiLyu
69d717fbc6ed9968ac5c69a1
Dandan Zheng
admin_assigned
2026-04-09T08:18:07.472Z
65dd699a89a2a760d15f7d35
/avatars/e098b56c413d147d1f38cf33a4b0ecde.svg
Dandan Zheng
zhengdd0422
user
zhengdd0422
69d717fbc6ed9968ac5c69a2
Chunhua Shen
69d717fbc6ed9968ac5c69a3
Jun Zhang
TC-AE: Unlocking Token Capacity for Deep Compression Autoencoders
We propose TC-AE, a ViT-based architecture for deep compression autoencoders. Existing methods commonly increase the channel number of latent representations to maintain reconstruction quality under high compression ratios. However, this strategy often leads to latent representation collapse, which degrades generative performance. Instead of relying on increasingly complex architectures or multi-stage training schemes, TC-AE addresses this challenge from the perspective of the token space, the key bridge between pixels and image latents, through two complementary innovations: Firstly, we study token number scaling by adjusting the patch size in ViT under a fixed latent budget, and identify aggressive token-to-latent compression as the key factor that limits effective scaling. To address this issue, we decompose token-to-latent compression into two stages, reducing structural information loss and enabling effective token number scaling for generation. Secondly, to further mitigate latent representation collapse, we enhance the semantic structure of image tokens via joint self-supervised training, leading to more generative-friendly latents. With these designs, TC-AE achieves substantially improved reconstruction and generative performance under deep compression. We hope our research will advance ViT-based tokenizer for visual generation.
TC-AE is a Vision Transformer-based architecture that improves deep compression autoencoders by addressing token space limitations and enhancing semantic structures through joint self-supervised training.
ViT-based architecture deep compression autoencoders latent representation collapse token space patch size token-to-latent compression joint self-supervised training visual generation