BitDance: Scaling Autoregressive Generative Models with Binary Tokens
We present BitDance, a scalable autoregressive (AR) image generator that predicts binary visual tokens instead of codebook indices. With high-entropy binary latents, BitDance lets each token represent up to 2^{256} states, yielding a compact yet highly expressive discrete representation. Sampling from such a huge token space is difficult with standard classification. To resolve this, BitDance uses a binary diffusion head: instead of predicting an index with softmax, it employs continuous-space diffusion to generate the binary tokens. Furthermore, we propose next-patch diffusion, a new decoding method that predicts multiple tokens in parallel with high accuracy, greatly speeding up inference. On ImageNet 256x256, BitDance achieves an FID of 1.24, the best among AR models. With next-patch diffusion, BitDance beats state-of-the-art parallel AR models that use 1.4B parameters, while using 5.4x fewer parameters (260M) and achieving 8.7x speedup. For text-to-image generation, BitDance trains on large-scale multimodal tokens and generates high-resolution, photorealistic images efficiently, showing strong performance and favorable scaling. When generating 1024x1024 images, BitDance achieves a speedup of over 30x compared to prior AR models. We release code and models to facilitate further research on AR foundation models. Code and models are available at: https://github.com/shallowdream204/BitDance.
6993d8d450fb2c0be4783ccc
Yuang Ai
claimed_verified
2026-02-18T09:06:59.779Z
65709464b3501cbcb8f4007e
https://cdn-avatars.huggingface.co/v1/production/uploads/65709464b3501cbcb8f4007e/lD6_mJvhiGPqYxIXhkKal.jpeg
Yuang Ai
shallowdream204
user
shallowdream204
6993d8d450fb2c0be4783ccd
Jiaming Han
claimed_verified
2026-02-18T09:06:57.914Z
62318c0386753f5f41d0e261
https://cdn-avatars.huggingface.co/v1/production/uploads/62318c0386753f5f41d0e261/xO_5PvOf7lXhQPnQLcmnq.jpeg
Jiaming Han
csuhan
user
csuhan
6993d8d450fb2c0be4783cce
Shaobin Zhuang
6993d8d450fb2c0be4783ccf
Weijia Mao
6993d8d450fb2c0be4783cd0
Xuefeng Hu
claimed_verified
2026-02-19T09:53:49.944Z
6830e8e81bdea85fad4c65f5
/avatars/f5aa39c61052c40240db8d42a35e6b52.svg
Xuefeng Hu
leonhuxff
user
leonhuxff
6993d8d450fb2c0be4783cd1
Ziyan Yang
6993d8d450fb2c0be4783cd2
Zhenheng Yang
6993d8d450fb2c0be4783cd3
Huaibo Huang
6993d8d450fb2c0be4783cd4
Xiangyu Yue
6993d8d450fb2c0be4783cd5
Hao Chen
BitDance: Scaling Autoregressive Generative Models with Binary Tokens
We present BitDance, a scalable autoregressive (AR) image generator that predicts binary visual tokens instead of codebook indices. With high-entropy binary latents, BitDance lets each token represent up to 2^{256} states, yielding a compact yet highly expressive discrete representation. Sampling from such a huge token space is difficult with standard classification. To resolve this, BitDance uses a binary diffusion head: instead of predicting an index with softmax, it employs continuous-space diffusion to generate the binary tokens. Furthermore, we propose next-patch diffusion, a new decoding method that predicts multiple tokens in parallel with high accuracy, greatly speeding up inference. On ImageNet 256x256, BitDance achieves an FID of 1.24, the best among AR models. With next-patch diffusion, BitDance beats state-of-the-art parallel AR models that use 1.4B parameters, while using 5.4x fewer parameters (260M) and achieving 8.7x speedup. For text-to-image generation, BitDance trains on large-scale multimodal tokens and generates high-resolution, photorealistic images efficiently, showing strong performance and favorable scaling. When generating 1024x1024 images, BitDance achieves a speedup of over 30x compared to prior AR models. We release code and models to facilitate further research on AR foundation models. Code and models are available at: https://github.com/shallowdream204/BitDance.
BitDance is a scalable autoregressive image generator that uses binary visual tokens and diffusion-based methods to achieve efficient high-resolution image generation with improved speed and performance.
autoregressive image generator binary visual tokens high-entropy binary latents binary diffusion head next-patch diffusion diffusion models FID parameter-efficient text-to-image generation photorealistic images image generation speedup