SimRecon: SimReady Compositional Scene Reconstruction from Real Videos
Compositional scene reconstruction seeks to create object-centric representations rather than holistic scenes from real-world videos, which is natively applicable for simulation and interaction. Conventional compositional reconstruction approaches primarily emphasize on visual appearance and show limited generalization ability to real-world scenarios. In this paper, we propose SimRecon, a framework that realizes a "Perception-Generation-Simulation" pipeline towards cluttered scene reconstruction, which first conducts scene-level semantic reconstruction from video input, then performs single-object generation, and finally assembles these assets in the simulator. However, naively combining these three stages leads to visual infidelity of generated assets and physical implausibility of the final scene, a problem particularly severe for complex scenes. Thus, we further propose two bridging modules between the three stages to address this problem. To be specific, for the transition from Perception to Generation, critical for visual fidelity, we introduce Active Viewpoint Optimization, which actively searches in 3D space to acquire optimal projected images as conditions for single-object completion. Moreover, for the transition from Generation to Simulation, essential for physical plausibility, we propose a Scene Graph Synthesizer, which guides the construction from scratch in 3D simulators, mirroring the native, constructive principle of the real world. Extensive experiments on the ScanNet dataset validate our method's superior performance over previous state-of-the-art approaches.
69b76f4691a28162b233fc12
Chong Xia
admin_assigned
2026-03-16T11:13:44.499Z
676c04f44464f476aaa53d1c
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/k488J1893F3JGwEMvaeuh.png
Chong Xia
xiac24
user
xiac24
69b76f4691a28162b233fc13
Kai Zhu
69b76f4691a28162b233fc14
Zizhuo Wang
admin_assigned
2026-03-16T11:13:55.765Z
690eb31aeed4fe1e4fa0daf7
/avatars/3ef17cb031fc9ce5fa1d7d43fbd25d5a.svg
Zizhuo Wang
Wzz234
user
Wzz234
69b76f4691a28162b233fc15
Fangfu Liu
admin_assigned
2026-03-16T11:14:01.951Z
6505a02f9310ce8c400edc63
/avatars/bbf781594fc8c812316711aa8e2797aa.svg
Fangfu Liu
Liuff23
user
Liuff23
69b76f4691a28162b233fc16
Zhizheng Zhang
69b76f4691a28162b233fc17
Yueqi Duan
admin_assigned
2026-03-16T11:14:11.512Z
66c8131afafc0fc87ca99650
/avatars/a6eeba2ccf011d5c9964fd38f85bd671.svg
Yueqi Duan
duanyueqi
user
duanyueqi
SimRecon: SimReady Compositional Scene Reconstruction from Real Videos
Compositional scene reconstruction seeks to create object-centric representations rather than holistic scenes from real-world videos, which is natively applicable for simulation and interaction. Conventional compositional reconstruction approaches primarily emphasize on visual appearance and show limited generalization ability to real-world scenarios. In this paper, we propose SimRecon, a framework that realizes a "Perception-Generation-Simulation" pipeline towards cluttered scene reconstruction, which first conducts scene-level semantic reconstruction from video input, then performs single-object generation, and finally assembles these assets in the simulator. However, naively combining these three stages leads to visual infidelity of generated assets and physical implausibility of the final scene, a problem particularly severe for complex scenes. Thus, we further propose two bridging modules between the three stages to address this problem. To be specific, for the transition from Perception to Generation, critical for visual fidelity, we introduce Active Viewpoint Optimization, which actively searches in 3D space to acquire optimal projected images as conditions for single-object completion. Moreover, for the transition from Generation to Simulation, essential for physical plausibility, we propose a Scene Graph Synthesizer, which guides the construction from scratch in 3D simulators, mirroring the native, constructive principle of the real world. Extensive experiments on the ScanNet dataset validate our method's superior performance over previous state-of-the-art approaches.
SimRecon enables compositional scene reconstruction through a perception-generation-simulation pipeline with active viewpoint optimization and scene graph synthesizer modules to improve visual fidelity and physical plausibility.
compositional scene reconstruction perception-generation-simulation pipeline active viewpoint optimization scene graph synthesizer 3D space 3D simulator ScanNet dataset