DiffusionVL: Translating Any Autoregressive Models into Diffusion Vision Language Models
In recent multimodal research, the diffusion paradigm has emerged as a promising alternative to the autoregressive paradigm (AR), owing to its unique decoding advantages. However, due to the capability limitations of the base diffusion language model, the performance of the diffusion vision language model (dVLM) still lags significantly behind that of mainstream models. This leads to a simple yet fundamental question: Is it possible to construct dVLMs based on existing powerful AR models? In response, we propose DiffusionVL, a dVLM family that could be translated from any powerful AR models. Through simple fine-tuning, we successfully adapt AR pre-trained models into the diffusion paradigm. This approach yields two key observations: (1) The paradigm shift from AR-based multimodal models to diffusion is remarkably effective. (2) Direct conversion of an AR language model to a dVLM is also feasible, achieving performance competitive with LLaVA-style visual-instruction-tuning. Further, we introduce a block-decoding design into dVLMs that supports arbitrary-length generation and KV cache reuse, achieving a significant inference speedup. We conduct a large number of experiments. Despite training with less than 5% of the data required by prior methods, DiffusionVL achieves a comprehensive performance improvement-a 34.4% gain on the MMMU-Pro (vision) bench and 37.5% gain on the MME (Cog.) bench-alongside a 2x inference speedup. The model and code are released at https://github.com/hustvl/DiffusionVL.
69437656542d62d58a7bf736
Lunbin Zeng
claimed_verified
2025-12-18T07:59:49.237Z
6375dfa7f9aafd41ce145254
/avatars/19e7460dd7ea6c60e1c52d5707660cc8.svg
Lunbin Zeng
xiazhi
user
xiazhi
69437656542d62d58a7bf737
Jingfeng Yao
69437656542d62d58a7bf738
Bencheng Liao
69437656542d62d58a7bf739
Hongyuan Tao
claimed_verified
2025-12-18T07:59:50.879Z
66a105bb456284adf458d656
/avatars/b543a324f7e159d6e84bc68915e93d24.svg
Tao Hongyuan
HongyuanTao
user
HongyuanTao
69437656542d62d58a7bf73a
Wenyu Liu
69437656542d62d58a7bf73b
Xinggang Wang
DiffusionVL: Translating Any Autoregressive Models into Diffusion Vision Language Models
In recent multimodal research, the diffusion paradigm has emerged as a promising alternative to the autoregressive paradigm (AR), owing to its unique decoding advantages. However, due to the capability limitations of the base diffusion language model, the performance of the diffusion vision language model (dVLM) still lags significantly behind that of mainstream models. This leads to a simple yet fundamental question: Is it possible to construct dVLMs based on existing powerful AR models? In response, we propose DiffusionVL, a dVLM family that could be translated from any powerful AR models. Through simple fine-tuning, we successfully adapt AR pre-trained models into the diffusion paradigm. This approach yields two key observations: (1) The paradigm shift from AR-based multimodal models to diffusion is remarkably effective. (2) Direct conversion of an AR language model to a dVLM is also feasible, achieving performance competitive with LLaVA-style visual-instruction-tuning. Further, we introduce a block-decoding design into dVLMs that supports arbitrary-length generation and KV cache reuse, achieving a significant inference speedup. We conduct a large number of experiments. Despite training with less than 5% of the data required by prior methods, DiffusionVL achieves a comprehensive performance improvement-a 34.4% gain on the MMMU-Pro (vision) bench and 37.5% gain on the MME (Cog.) bench-alongside a 2x inference speedup. The model and code are released at https://github.com/hustvl/DiffusionVL.
DiffusionVL, a family of diffusion vision language models derived from autoregressive models through fine-tuning, achieves performance improvements and faster inference speeds compared to existing models.
diffusion paradigm autoregressive paradigm diffusion vision language model dVLM DiffusionVL fine-tuning block-decoding design KV cache reuse MMMU-Pro MME Cog.