LLaDA2.0: Scaling Up Diffusion Language Models to 100B
This paper presents LLaDA2.0 -- a tuple of discrete diffusion large language models (dLLM) scaling up to 100B total parameters through systematic conversion from auto-regressive (AR) models -- establishing a new paradigm for frontier-scale deployment. Instead of costly training from scratch, LLaDA2.0 upholds knowledge inheritance, progressive adaption and efficiency-aware design principle, and seamless converts a pre-trained AR model into dLLM with a novel 3-phase block-level WSD based training scheme: progressive increasing block-size in block diffusion (warm-up), large-scale full-sequence diffusion (stable) and reverting back to compact-size block diffusion (decay). Along with post-training alignment with SFT and DPO, we obtain LLaDA2.0-mini (16B) and LLaDA2.0-flash (100B), two instruction-tuned Mixture-of-Experts (MoE) variants optimized for practical deployment. By preserving the advantages of parallel decoding, these models deliver superior performance and efficiency at the frontier scale. Both models were open-sourced.
6944bdd4fbf17e708e185fb8
Tiwei Bie
6944bdd4fbf17e708e185fb9
Maosong Cao
6944bdd4fbf17e708e185fba
Kun Chen
6944bdd4fbf17e708e185fbb
Lun Du
6944bdd4fbf17e708e185fbc
Mingliang Gong
admin_assigned
2025-12-19T09:49:22.650Z
6909e9153e33b19bf2f71b05
/avatars/8fa060fa5c21ce57c2e5f87a0835af07.svg
Mingliang Gong
bright-ai-infra
user
bright-ai-infra
6944bdd4fbf17e708e185fbd
Zhuochen Gong
claimed_verified
2025-12-19T09:56:22.280Z
64a68e21d09682887d9ed95a
/avatars/2d7be7d9221b7a59ecbeb5383f70d83d.svg
Zhuocheng Gong
gzhch
user
gzhch
6944bdd4fbf17e708e185fbe
Yanmei Gu
6944bdd4fbf17e708e185fbf
Jiaqi Hu
6944bdd4fbf17e708e185fc0
Zenan Huang
claimed_verified
2025-12-19T09:56:17.788Z
625f8e6f673e5862a8c07f1a
https://cdn-avatars.huggingface.co/v1/production/uploads/1677306231208-625f8e6f673e5862a8c07f1a.jpeg
Bill H
lccurious
user
lccurious
6944bdd4fbf17e708e185fc1
Zhenzhong Lan
6944bdd4fbf17e708e185fc2
Chengxi Li
claimed_verified
2025-12-25T20:52:35.708Z
6486e362c9d77c91be1ee2bf
/avatars/c8a3f4d0cf59676f0ae0f49f5e478670.svg
Chengxi Li
MrZixi
user
MrZixi
6944bdd4fbf17e708e185fc3
Chongxuan Li
admin_assigned
2025-12-19T09:56:49.796Z
64c07b488e2612254361153b
/avatars/ade0f783cc4c2d3e73f402637f595471.svg
chongxuan li
zhenxuan00
user
zhenxuan00
6944bdd4fbf17e708e185fc4
Jianguo Li
claimed_verified
2025-12-25T20:52:38.964Z
6539fc3fe279d2fc80c88902
/avatars/432ac2287a7f7d04c2c7e505070a3331.svg
Jianguo Li
jglee2046
user
jglee2046
6944bdd4fbf17e708e185fc5
Zehuan Li
6944bdd4fbf17e708e185fc6
Huabin Liu
claimed_verified
2025-12-22T11:00:31.684Z
6470600790482b0e0f65a393
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/fOUWWhf9jjsC9fgIiFlPc.png
Huabin
Rookie-Liu
user
Rookie-Liu
6944bdd4fbf17e708e185fc7
Ling Liu
claimed_verified
2025-12-19T10:49:51.184Z
646ed6708d316fde87b3eee3
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/6PjHGz3w2iPYa8eZCP2Qc.jpeg
Liulin
Ulov888
user
Ulov888
6944bdd4fbf17e708e185fc8
Guoshan Lu
claimed_verified
2025-12-19T09:56:23.767Z
690077c498a1389fbb6e9a5d
https://cdn-avatars.huggingface.co/v1/production/uploads/690077c498a1389fbb6e9a5d/JPf4OH46m0uU7CH9aO4Yo.jpeg
Guoshan Lu
luguoshan
user
luguoshan
6944bdd4fbf17e708e185fc9
Xiaocheng Lu
6944bdd4fbf17e708e185fca
Yuxin Ma
6944bdd4fbf17e708e185fcb
Jianfeng Tan
admin_assigned
2025-12-19T09:57:09.384Z
64d2eaee80ce1029be72bdcd
/avatars/6917aa66459d6f9a4d6437381c496bdd.svg
Jianfeng Tan
jianfengt
user
jianfengt
6944bdd4fbf17e708e185fcc
Lanning Wei
6944bdd4fbf17e708e185fcd
Ji-Rong Wen
admin_assigned
2025-12-19T09:57:23.843Z
64b8c89052b7353d8c6a1013
/avatars/cd59fffe81f6b07b4519540b8ff3d95f.svg
Ji-Rong Wen
jrwen
user
jrwen
6944bdd4fbf17e708e185fce
Yipeng Xing
6944bdd4fbf17e708e185fcf
Xiaolu Zhang
6944bdd4fbf17e708e185fd0
Junbo Zhao
claimed_verified
2025-12-19T09:56:20.782Z
6725f5a7f05f62659e3615f3
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/iiXdzhaSnExw2dwQQtLZ8.png
Junbo Zhao
jakezhao2024
user
jakezhao2024
6944bdd4fbf17e708e185fd1
Da Zheng
6944bdd4fbf17e708e185fd2
Jun Zhou
6944bdd4fbf17e708e185fd3
Junlin Zhou
admin_assigned
2025-12-19T12:13:31.053Z
63eb008e5c837d9968f1eb71
/avatars/ae43c3f5ab87b82f4bad25c65ac55d01.svg
Junlin Zhou
jlzhou
user
jlzhou
6944bdd4fbf17e708e185fd4
Zhanchao Zhou
claimed_verified
2026-02-11T11:21:00.500Z
671b3d82101c0d9f78749256
/avatars/902c1006280c6552885b0b946cd2bb16.svg
Zhanchao Zhou
Zcchill
user
Zcchill
6944bdd4fbf17e708e185fd5
Liwang Zhu
6944bdd4fbf17e708e185fd6
Yihong Zhuang
claimed_verified
2025-12-19T09:56:19.300Z
673b5f24e863f1d28b402efc
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/19gUgtPEY3-FtY0sNlI_-.png
yihongzhuang
utdawn
user
utdawn
LLaDA2.0: Scaling Up Diffusion Language Models to 100B
This paper presents LLaDA2.0 -- a tuple of discrete diffusion large language models (dLLM) scaling up to 100B total parameters through systematic conversion from auto-regressive (AR) models -- establishing a new paradigm for frontier-scale deployment. Instead of costly training from scratch, LLaDA2.0 upholds knowledge inheritance, progressive adaption and efficiency-aware design principle, and seamless converts a pre-trained AR model into dLLM with a novel 3-phase block-level WSD based training scheme: progressive increasing block-size in block diffusion (warm-up), large-scale full-sequence diffusion (stable) and reverting back to compact-size block diffusion (decay). Along with post-training alignment with SFT and DPO, we obtain LLaDA2.0-mini (16B) and LLaDA2.0-flash (100B), two instruction-tuned Mixture-of-Experts (MoE) variants optimized for practical deployment. By preserving the advantages of parallel decoding, these models deliver superior performance and efficiency at the frontier scale. Both models were open-sourced.
LLaDA2.0 converts auto-regressive models into discrete diffusion large language models with a novel training scheme, achieving superior performance and efficiency at scale.
discrete diffusion large language models dLLM auto-regressive models knowledge inheritance progressive adaptation efficiency-aware design block-level WSD block diffusion full-sequence diffusion post-training alignment SFT DPO Mixture-of-Experts parallel decoding