Are We Ready for RL in Text-to-3D Generation? A Progressive Investigation
Reinforcement learning (RL), earlier proven to be effective in large language and multi-modal models, has been successfully extended to enhance 2D image generation recently. However, applying RL to 3D generation remains largely unexplored due to the higher spatial complexity of 3D objects, which require globally consistent geometry and fine-grained local textures. This makes 3D generation significantly sensitive to reward designs and RL algorithms. To address these challenges, we conduct the first systematic study of RL for text-to-3D autoregressive generation across several dimensions. (1) Reward designs: We evaluate reward dimensions and model choices, showing that alignment with human preference is crucial, and that general multi-modal models provide robust signal for 3D attributes. (2) RL algorithms: We study GRPO variants, highlighting the effectiveness of token-level optimization, and further investigate the scaling of training data and iterations. (3) Text-to-3D Benchmarks: Since existing benchmarks fail to measure implicit reasoning abilities in 3D generation models, we introduce MME-3DR. (4) Advanced RL paradigms: Motivated by the natural hierarchy of 3D generation, we propose Hi-GRPO, which optimizes the global-to-local hierarchical 3D generation through dedicated reward ensembles. Based on these insights, we develop AR3D-R1, the first RL-enhanced text-to-3D model, expert from coarse shape to texture refinement. We hope this study provides insights into RL-driven reasoning for 3D generation. Code is released at https://github.com/Ivan-Tang-3D/3DGen-R1.
693b895d9874a2a5e4ffb302
Yiwen Tang
admin_assigned
2025-12-12T13:04:55.504Z
6552f1ad5d55ccb20e9142a0
/avatars/0e3e80cba64b5ae0bc5638694ac33dbf.svg
Ivan Tang
IvanTang
user
IvanTang
693b895d9874a2a5e4ffb303
Zoey Guo
admin_assigned
2025-12-12T13:05:01.705Z
642a8302d651bae3c11b72b1
/avatars/4d2d422613e274d80482fed9a7d3f785.svg
Zoey Guo
Purple1288
user
Purple1288
693b895d9874a2a5e4ffb304
Kaixin Zhu
claimed_verified
2025-12-12T09:16:10.275Z
6708920aeae29d1cd41a703b
/avatars/922427a86523b0aa810412fd2d75f88e.svg
kaixin zhu
czkk566
user
czkk566
693b895d9874a2a5e4ffb305
Ray Zhang
693b895d9874a2a5e4ffb306
Qizhi Chen
admin_assigned
2025-12-12T13:05:11.060Z
6535045a910b844786a6642f
/avatars/37a94864a7a348151837b421ea6d77e3.svg
Qizhi Chen
Tavish9
user
Tavish9
693b895d9874a2a5e4ffb307
Dongzhi Jiang
admin_assigned
2025-12-12T13:06:02.910Z
6349214f8146350b3a4c5cdf
/avatars/cfd24caac9a87efb528d0f4c375932bc.svg
Dongzhi Jiang
CaraJ
user
CaraJ
693b895d9874a2a5e4ffb308
Junli Liu
693b895d9874a2a5e4ffb309
Bohan Zeng
claimed_verified
2025-12-12T09:15:08.733Z
6671214c92412fd4640714eb
/avatars/48fa84e7bc3bb92ad0192aa26b32de10.svg
bohan zeng
zbhpku
user
zbhpku
693b895d9874a2a5e4ffb30a
Haoming Song
admin_assigned
2025-12-12T13:05:30.062Z
6662d2c9de4c4e1f04bd29c7
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/QnnO_KOyZjFd-iXuPnHqR.png
HaomingSong
HaomingSong
user
HaomingSong
693b895d9874a2a5e4ffb30b
Delin Qu
claimed_verified
2025-12-12T09:16:34.085Z
64daecec888b7e9c400f59b5
https://cdn-avatars.huggingface.co/v1/production/uploads/64daecec888b7e9c400f59b5/DuSAooSj4sKAp1sT_jHFU.png
Delin Qu
delinqu
user
delinqu
693b895d9874a2a5e4ffb30c
Tianyi Bai
693b895d9874a2a5e4ffb30d
Dan Xu
claimed_verified
2025-12-15T08:14:00.909Z
66feab48651e00e22f33222e
/avatars/7344377e2c796c7ec85194bb2fc78521.svg
Dan Xu
danxuhk
user
danxuhk
693b895d9874a2a5e4ffb30e
Wentao Zhang
693b895d9874a2a5e4ffb30f
Bin Zhao
Are We Ready for RL in Text-to-3D Generation? A Progressive Investigation
Reinforcement learning (RL), earlier proven to be effective in large language and multi-modal models, has been successfully extended to enhance 2D image generation recently. However, applying RL to 3D generation remains largely unexplored due to the higher spatial complexity of 3D objects, which require globally consistent geometry and fine-grained local textures. This makes 3D generation significantly sensitive to reward designs and RL algorithms. To address these challenges, we conduct the first systematic study of RL for text-to-3D autoregressive generation across several dimensions. (1) Reward designs: We evaluate reward dimensions and model choices, showing that alignment with human preference is crucial, and that general multi-modal models provide robust signal for 3D attributes. (2) RL algorithms: We study GRPO variants, highlighting the effectiveness of token-level optimization, and further investigate the scaling of training data and iterations. (3) Text-to-3D Benchmarks: Since existing benchmarks fail to measure implicit reasoning abilities in 3D generation models, we introduce MME-3DR. (4) Advanced RL paradigms: Motivated by the natural hierarchy of 3D generation, we propose Hi-GRPO, which optimizes the global-to-local hierarchical 3D generation through dedicated reward ensembles. Based on these insights, we develop AR3D-R1, the first RL-enhanced text-to-3D model, expert from coarse shape to texture refinement. We hope this study provides insights into RL-driven reasoning for 3D generation. Code is released at https://github.com/Ivan-Tang-3D/3DGen-R1.
This study investigates reinforcement learning for text-to-3D generation, focusing on reward designs, RL algorithms, benchmarking, and hierarchical optimization, introducing AR3D-R1 as the first RL-enhanced model for 3D generation.
reinforcement learning text-to-3D generation reward designs GRPO variants token-level optimization MME-3DR Hi-GRPO hierarchical 3D generation AR3D-R1