DiRL: An Efficient Post-Training Framework for Diffusion Language Models
Diffusion Language Models (dLLMs) have emerged as promising alternatives to Auto-Regressive (AR) models. While recent efforts have validated their pre-training potential and accelerated inference speeds, the post-training landscape for dLLMs remains underdeveloped. Existing methods suffer from computational inefficiency and objective mismatches between training and inference, severely limiting performance on complex reasoning tasks such as mathematics. To address this, we introduce DiRL, an efficient post-training framework that tightly integrates FlexAttention-accelerated blockwise training with LMDeploy-optimized inference. This architecture enables a streamlined online model update loop, facilitating efficient two-stage post-training (Supervised Fine-Tuning followed by Reinforcement Learning). Building on this framework, we propose DiPO, the first unbiased Group Relative Policy Optimization (GRPO) implementation tailored for dLLMs. We validate our approach by training DiRL-8B-Instruct on high-quality math data. Our model achieves state-of-the-art math performance among dLLMs and surpasses comparable models in the Qwen2.5 series on several benchmarks.
695364c389916ff627aa404f
Ying Zhu
claimed_verified
2025-12-31T20:56:20.416Z
69158ffc0153b85a677dcc46
/avatars/c9c5f60522f2a8f370d790ea9938b090.svg
Ying Zhu
Auraithm
user
Auraithm
695364c389916ff627aa4050
Jiaxin Wan
695364c389916ff627aa4051
Xiaoran Liu
claimed_verified
2025-12-31T20:56:18.599Z
64f033ef82c6eea604c4da8b
/avatars/51b93fea7fd68b4274ee03701245dcca.svg
Xiaoran Liu (SII)
SII-xrliu
user
SII-xrliu
695364c389916ff627aa4052
Siyanag He
695364c389916ff627aa4053
Qiqi Wang
695364c389916ff627aa4054
Xu Guo
695364c389916ff627aa4055
Tianyi Liang
claimed_verified
2026-01-04T20:31:57.172Z
62c14609ac1b639c2d87192c
https://cdn-avatars.huggingface.co/v1/production/uploads/1656833489364-noauth.png
SII-liangtianyi
tianyilt
user
tianyilt
695364c389916ff627aa4056
Zengfeng Huang
695364c389916ff627aa4057
Ziwei He
695364c389916ff627aa4058
Xipeng Qiu
DiRL: An Efficient Post-Training Framework for Diffusion Language Models
Diffusion Language Models (dLLMs) have emerged as promising alternatives to Auto-Regressive (AR) models. While recent efforts have validated their pre-training potential and accelerated inference speeds, the post-training landscape for dLLMs remains underdeveloped. Existing methods suffer from computational inefficiency and objective mismatches between training and inference, severely limiting performance on complex reasoning tasks such as mathematics. To address this, we introduce DiRL, an efficient post-training framework that tightly integrates FlexAttention-accelerated blockwise training with LMDeploy-optimized inference. This architecture enables a streamlined online model update loop, facilitating efficient two-stage post-training (Supervised Fine-Tuning followed by Reinforcement Learning). Building on this framework, we propose DiPO, the first unbiased Group Relative Policy Optimization (GRPO) implementation tailored for dLLMs. We validate our approach by training DiRL-8B-Instruct on high-quality math data. Our model achieves state-of-the-art math performance among dLLMs and surpasses comparable models in the Qwen2.5 series on several benchmarks.