Real2Edit2Real: Generating Robotic Demonstrations via a 3D Control Interface
Recent progress in robot learning has been driven by large-scale datasets and powerful visuomotor policy architectures, yet policy robustness remains limited by the substantial cost of collecting diverse demonstrations, particularly for spatial generalization in manipulation tasks. To reduce repetitive data collection, we present Real2Edit2Real, a framework that generates new demonstrations by bridging 3D editability with 2D visual data through a 3D control interface. Our approach first reconstructs scene geometry from multi-view RGB observations with a metric-scale 3D reconstruction model. Based on the reconstructed geometry, we perform depth-reliable 3D editing on point clouds to generate new manipulation trajectories while geometrically correcting the robot poses to recover physically consistent depth, which serves as a reliable condition for synthesizing new demonstrations. Finally, we propose a multi-conditional video generation model guided by depth as the primary control signal, together with action, edge, and ray maps, to synthesize spatially augmented multi-view manipulation videos. Experiments on four real-world manipulation tasks demonstrate that policies trained on data generated from only 1-5 source demonstrations can match or outperform those trained on 50 real-world demonstrations, improving data efficiency by up to 10-50x. Moreover, experimental results on height and texture editing demonstrate the framework's flexibility and extensibility, indicating its potential to serve as a unified data generation framework.
694a1f4e335742716e9322d0
Yujie Zhao
claimed_verified
2025-12-25T20:48:05.673Z
6721acf7bbf9703bc285e840
/avatars/487f5cae476758d318465aa9ed103568.svg
Yujie Zhao
HomieZ
user
HomieZ
694a1f4e335742716e9322d1
Hongwei Fan
694a1f4e335742716e9322d2
Di Chen
694a1f4e335742716e9322d3
Shengcong Chen
694a1f4e335742716e9322d4
Liliang Chen
694a1f4e335742716e9322d5
Xiaoqi Li
694a1f4e335742716e9322d6
Guanghui Ren
claimed_verified
2026-03-10T09:46:44.044Z
646ec9b135f55eb49e405faa
/avatars/a17194be585d20e2a021e77a5a20e213.svg
Guanghui Ren
sundrops
user
sundrops
694a1f4e335742716e9322d7
Hao Dong
Real2Edit2Real: Generating Robotic Demonstrations via a 3D Control Interface
Recent progress in robot learning has been driven by large-scale datasets and powerful visuomotor policy architectures, yet policy robustness remains limited by the substantial cost of collecting diverse demonstrations, particularly for spatial generalization in manipulation tasks. To reduce repetitive data collection, we present Real2Edit2Real, a framework that generates new demonstrations by bridging 3D editability with 2D visual data through a 3D control interface. Our approach first reconstructs scene geometry from multi-view RGB observations with a metric-scale 3D reconstruction model. Based on the reconstructed geometry, we perform depth-reliable 3D editing on point clouds to generate new manipulation trajectories while geometrically correcting the robot poses to recover physically consistent depth, which serves as a reliable condition for synthesizing new demonstrations. Finally, we propose a multi-conditional video generation model guided by depth as the primary control signal, together with action, edge, and ray maps, to synthesize spatially augmented multi-view manipulation videos. Experiments on four real-world manipulation tasks demonstrate that policies trained on data generated from only 1-5 source demonstrations can match or outperform those trained on 50 real-world demonstrations, improving data efficiency by up to 10-50x. Moreover, experimental results on height and texture editing demonstrate the framework's flexibility and extensibility, indicating its potential to serve as a unified data generation framework.
A framework called Real2Edit2Real generates new manipulation demonstrations by using 3D reconstruction, editing, and video synthesis, improving data efficiency in robot learning.
metric-scale 3D reconstruction model depth-reliable 3D editing multi-conditional video generation model depth action edge ray maps spatially augmented multi-view manipulation videos