Cubic Discrete Diffusion: Discrete Visual Generation on High-Dimensional Representation Tokens
Visual generation with discrete tokens has gained significant attention as it enables a unified token prediction paradigm shared with language models, promising seamless multimodal architectures. However, current discrete generation methods remain limited to low-dimensional latent tokens (typically 8-32 dims), sacrificing the semantic richness essential for understanding. While high-dimensional pretrained representations (768-1024 dims) could bridge this gap, their discrete generation poses fundamental challenges. In this paper, we present Cubic Discrete Diffusion (CubiD), the first discrete generation model for high-dimensional representations. CubiD performs fine-grained masking throughout the high-dimensional discrete representation -- any dimension at any position can be masked and predicted from partial observations. This enables the model to learn rich correlations both within and across spatial positions, with the number of generation steps fixed at T regardless of feature dimensionality, where T ll hwd. On ImageNet-256, CubiD achieves state-of-the-art discrete generation with strong scaling behavior from 900M to 3.7B parameters. Crucially, we validate that these discretized tokens preserve original representation capabilities, demonstrating that the same discrete tokens can effectively serve both understanding and generation tasks. We hope this work will inspire future research toward unified multimodal architectures. Code is available at: https://github.com/YuqingWang1029/CubiD.
69bcad20081b9ad49af571bd
Yuqing Wang
claimed_verified
2026-03-23T10:55:03.650Z
63ea23b9dedfeebe54d02bdf
/avatars/4d9f9a546aa8c63e277161ea700075c4.svg
Yuqing Wang
Epiphqny
user
Epiphqny
69bcad20081b9ad49af571be
Chuofan Ma
69bcad20081b9ad49af571bf
Zhijie Lin
69bcad20081b9ad49af571c0
Yao Teng
69bcad20081b9ad49af571c1
Lijun Yu
69bcad20081b9ad49af571c2
Shuai Wang
claimed_verified
2026-04-02T15:32:24.150Z
66615c855fd9d736e670e0a9
/avatars/0ff3127b513552432a7c651e21d7f283.svg
wangshuai
wangsssssss
user
wangsssssss
69bcad20081b9ad49af571c3
Jiaming Han
69bcad20081b9ad49af571c4
Jiashi Feng
69bcad20081b9ad49af571c5
Yi Jiang
69bcad20081b9ad49af571c6
Xihui Liu
Cubic Discrete Diffusion: Discrete Visual Generation on High-Dimensional Representation Tokens
Visual generation with discrete tokens has gained significant attention as it enables a unified token prediction paradigm shared with language models, promising seamless multimodal architectures. However, current discrete generation methods remain limited to low-dimensional latent tokens (typically 8-32 dims), sacrificing the semantic richness essential for understanding. While high-dimensional pretrained representations (768-1024 dims) could bridge this gap, their discrete generation poses fundamental challenges. In this paper, we present Cubic Discrete Diffusion (CubiD), the first discrete generation model for high-dimensional representations. CubiD performs fine-grained masking throughout the high-dimensional discrete representation -- any dimension at any position can be masked and predicted from partial observations. This enables the model to learn rich correlations both within and across spatial positions, with the number of generation steps fixed at T regardless of feature dimensionality, where T ll hwd. On ImageNet-256, CubiD achieves state-of-the-art discrete generation with strong scaling behavior from 900M to 3.7B parameters. Crucially, we validate that these discretized tokens preserve original representation capabilities, demonstrating that the same discrete tokens can effectively serve both understanding and generation tasks. We hope this work will inspire future research toward unified multimodal architectures. Code is available at: https://github.com/YuqingWang1029/CubiD.
CubiD is a discrete generation model for high-dimensional representations that enables fine-grained masking and learns rich correlations across spatial positions while maintaining fixed generation steps regardless of feature dimensionality.
discrete generation high-dimensional representations fine-grained masking spatial positions generation steps discrete diffusion token prediction multimodal architectures pretrained representations visual generation