Geometry-Guided Reinforcement Learning for Multi-view Consistent 3D Scene Editing
Leveraging the priors of 2D diffusion models for 3D editing has emerged as a promising paradigm. However, maintaining multi-view consistency in edited results remains challenging, and the extreme scarcity of 3D-consistent editing paired data renders supervised fine-tuning (SFT), the most effective training strategy for editing tasks, infeasible. In this paper, we observe that, while generating multi-view consistent 3D content is highly challenging, verifying 3D consistency is tractable, naturally positioning reinforcement learning (RL) as a feasible solution. Motivated by this, we propose RL3DEdit, a single-pass framework driven by RL optimization with novel rewards derived from the 3D foundation model, VGGT. Specifically, we leverage VGGT's robust priors learned from massive real-world data, feed the edited images, and utilize the output confidence maps and pose estimation errors as reward signals, effectively anchoring the 2D editing priors onto a 3D-consistent manifold via RL. Extensive experiments demonstrate that RL3DEdit achieves stable multi-view consistency and outperforms state-of-the-art methods in editing quality with high efficiency. To promote the development of 3D editing, we will release the code and model.
69b0e8251f7902d47f6dfa71
Jiyuan Wang
claimed_verified
2026-03-11T08:28:28.746Z
63cf4ceb7332dafae2b9884c
/avatars/652255476e1898c175c28f7a1d67cc06.svg
JiYuan Wang
exander
user
exander
69b0e8251f7902d47f6dfa72
Chunyu Lin
69b0e8251f7902d47f6dfa73
Lei Sun
69b0e8251f7902d47f6dfa74
Zhi Cao
69b0e8251f7902d47f6dfa75
Yuyang Yin
admin_assigned
2026-03-11T08:52:57.043Z
64b7ab4c037d6452a31910eb
https://cdn-avatars.huggingface.co/v1/production/uploads/64b7ab4c037d6452a31910eb/0UaBtwyQTysBMndFWZdKu.png
yuyangyin
yuyangyin
user
yuyangyin
69b0e8251f7902d47f6dfa76
Lang Nie
claimed_verified
2026-03-11T08:44:18.624Z
68ba95c4a44514e1054952e1
/avatars/6bba5a1457a514bec32c34d20fbc7976.svg
Lang Nie
lang96
user
lang96
69b0e8251f7902d47f6dfa77
Zhenlong Yuan
admin_assigned
2026-03-11T08:52:45.027Z
693d4ec248332487069f8004
/avatars/c27ff0cf7683717cc86200d7d0d0ec3d.svg
Zhenlong Yuan
ZhenlongYuan
user
ZhenlongYuan
69b0e8251f7902d47f6dfa78
Xiangxiang Chu
69b0e8251f7902d47f6dfa79
Yunchao Wei
69b0e8251f7902d47f6dfa7a
Kang Liao
admin_assigned
2026-03-11T08:51:44.119Z
65bc98383b879593a5a2f5e5
https://cdn-avatars.huggingface.co/v1/production/uploads/65bc98383b879593a5a2f5e5/p2ZtoTFN6tW-QkcPJf7YT.jpeg
Kang Liao
KangLiao
user
KangLiao
69b0e8251f7902d47f6dfa7b
Guosheng Lin
Geometry-Guided Reinforcement Learning for Multi-view Consistent 3D Scene Editing
Leveraging the priors of 2D diffusion models for 3D editing has emerged as a promising paradigm. However, maintaining multi-view consistency in edited results remains challenging, and the extreme scarcity of 3D-consistent editing paired data renders supervised fine-tuning (SFT), the most effective training strategy for editing tasks, infeasible. In this paper, we observe that, while generating multi-view consistent 3D content is highly challenging, verifying 3D consistency is tractable, naturally positioning reinforcement learning (RL) as a feasible solution. Motivated by this, we propose RL3DEdit, a single-pass framework driven by RL optimization with novel rewards derived from the 3D foundation model, VGGT. Specifically, we leverage VGGT's robust priors learned from massive real-world data, feed the edited images, and utilize the output confidence maps and pose estimation errors as reward signals, effectively anchoring the 2D editing priors onto a 3D-consistent manifold via RL. Extensive experiments demonstrate that RL3DEdit achieves stable multi-view consistency and outperforms state-of-the-art methods in editing quality with high efficiency. To promote the development of 3D editing, we will release the code and model.
RL3DEdit uses reinforcement learning with rewards from a 3D foundation model to achieve multi-view consistent 3D editing from 2D editing priors.
diffusion models reinforcement learning 3D editing multi-view consistency supervised fine-tuning VGGT reward signals 3D foundation model