SNCE: Geometry-Aware Supervision for Scalable Discrete Image Generation
Recent advancements in discrete image generation showed that scaling the VQ codebook size significantly improves reconstruction fidelity. However, training generative models with a large VQ codebook remains challenging, typically requiring larger model size and a longer training schedule. In this work, we propose Stochastic Neighbor Cross Entropy Minimization (SNCE), a novel training objective designed to address the optimization challenges of large-codebook discrete image generators. Instead of supervising the model with a hard one-hot target, SNCE constructs a soft categorical distribution over a set of neighboring tokens. The probability assigned to each token is proportional to the proximity between its code embedding and the ground-truth image embedding, encouraging the model to capture semantically meaningful geometric structure in the quantized embedding space. We conduct extensive experiments across class-conditional ImageNet-256 generation, large-scale text-to-image synthesis, and image editing tasks. Results show that SNCE significantly improves convergence speed and overall generation quality compared to standard cross-entropy objectives.
69b8f2046b3f36c249fbc52b
Shufan Li
admin_assigned
2026-03-17T09:54:06.823Z
6310531914aa81e1044363ed
/avatars/ae7767e591cb7199ea2f62d2db89fc7f.svg
Shufan Li
jacklishufan
user
jacklishufan
69b8f2046b3f36c249fbc52c
Jiuxiang Gu
admin_assigned
2026-03-17T09:54:13.427Z
642467708d97ce93878e8124
/avatars/f0e464ddb4bd790f470fc0f10275fa26.svg
Jiuxiang Gu
JoshuaGu
user
JoshuaGu
69b8f2046b3f36c249fbc52d
Kangning Liu
admin_assigned
2026-03-17T09:54:22.481Z
6711e867f25bb4f32f880872
/avatars/02ec6a9e59f4c4eec6cc7142939cea79.svg
Kangning Liu
kl3141
user
kl3141
69b8f2046b3f36c249fbc52e
Zhe Lin
69b8f2046b3f36c249fbc52f
Aditya Grover
69b8f2046b3f36c249fbc530
Jason Kuen
admin_assigned
2026-03-17T09:54:32.047Z
65384eaeef7d2bb8c4baf1b2
/avatars/b9217561a59309d6b0ddb417324c6cad.svg
Jason Kuen
xternalz
user
xternalz
SNCE: Geometry-Aware Supervision for Scalable Discrete Image Generation
Recent advancements in discrete image generation showed that scaling the VQ codebook size significantly improves reconstruction fidelity. However, training generative models with a large VQ codebook remains challenging, typically requiring larger model size and a longer training schedule. In this work, we propose Stochastic Neighbor Cross Entropy Minimization (SNCE), a novel training objective designed to address the optimization challenges of large-codebook discrete image generators. Instead of supervising the model with a hard one-hot target, SNCE constructs a soft categorical distribution over a set of neighboring tokens. The probability assigned to each token is proportional to the proximity between its code embedding and the ground-truth image embedding, encouraging the model to capture semantically meaningful geometric structure in the quantized embedding space. We conduct extensive experiments across class-conditional ImageNet-256 generation, large-scale text-to-image synthesis, and image editing tasks. Results show that SNCE significantly improves convergence speed and overall generation quality compared to standard cross-entropy objectives.