Autoregressive Image Generation with Masked Bit Modeling
This paper challenges the dominance of continuous pipelines in visual generation. We systematically investigate the performance gap between discrete and continuous methods. Contrary to the belief that discrete tokenizers are intrinsically inferior, we demonstrate that the disparity arises primarily from the total number of bits allocated in the latent space (i.e., the compression ratio). We show that scaling up the codebook size effectively bridges this gap, allowing discrete tokenizers to match or surpass their continuous counterparts. However, existing discrete generation methods struggle to capitalize on this insight, suffering from performance degradation or prohibitive training costs with scaled codebook. To address this, we propose masked Bit AutoRegressive modeling (BAR), a scalable framework that supports arbitrary codebook sizes. By equipping an autoregressive transformer with a masked bit modeling head, BAR predicts discrete tokens through progressively generating their constituent bits. BAR achieves a new state-of-the-art gFID of 0.99 on ImageNet-256, outperforming leading methods across both continuous and discrete paradigms, while significantly reducing sampling costs and converging faster than prior continuous approaches. Project page is available at https://bar-gen.github.io/
698ab8ae1b2dc6b37d61b089
Qihang Yu
claimed_verified
2026-03-31T09:31:57.225Z
648010b00b9d0f49849adb19
/avatars/54419bb6e6ee788aba10cd64cd921204.svg
Qihang Yu
yucornetto
user
yucornetto
698ab8ae1b2dc6b37d61b08a
Qihao Liu
698ab8ae1b2dc6b37d61b08b
Ju He
claimed_verified
2026-02-12T20:28:47.491Z
661c9059bcd78151e5c06ea1
https://cdn-avatars.huggingface.co/v1/production/uploads/661c9059bcd78151e5c06ea1/27bfNo1LZeZQ77vWuAa10.png
Ju He
turkeyju
user
turkeyju
698ab8ae1b2dc6b37d61b08c
Xinyang Zhang
698ab8ae1b2dc6b37d61b08d
Yang Liu
698ab8ae1b2dc6b37d61b08e
Liang-Chieh Chen
698ab8ae1b2dc6b37d61b08f
Xi Chen
Autoregressive Image Generation with Masked Bit Modeling
This paper challenges the dominance of continuous pipelines in visual generation. We systematically investigate the performance gap between discrete and continuous methods. Contrary to the belief that discrete tokenizers are intrinsically inferior, we demonstrate that the disparity arises primarily from the total number of bits allocated in the latent space (i.e., the compression ratio). We show that scaling up the codebook size effectively bridges this gap, allowing discrete tokenizers to match or surpass their continuous counterparts. However, existing discrete generation methods struggle to capitalize on this insight, suffering from performance degradation or prohibitive training costs with scaled codebook. To address this, we propose masked Bit AutoRegressive modeling (BAR), a scalable framework that supports arbitrary codebook sizes. By equipping an autoregressive transformer with a masked bit modeling head, BAR predicts discrete tokens through progressively generating their constituent bits. BAR achieves a new state-of-the-art gFID of 0.99 on ImageNet-256, outperforming leading methods across both continuous and discrete paradigms, while significantly reducing sampling costs and converging faster than prior continuous approaches. Project page is available at https://bar-gen.github.io/
Discrete tokenizers can match or exceed continuous methods when properly scaled, and a new masked Bit AutoRegressive modeling approach achieves state-of-the-art results with reduced computational costs.
discrete tokenizers continuous pipelines latent space codebook size autoregressive transformer masked bit modeling Bit AutoRegressive modeling gFID ImageNet-256