Gen3R: 3D Scene Generation Meets Feed-Forward Reconstruction
We present Gen3R, a method that bridges the strong priors of foundational reconstruction models and video diffusion models for scene-level 3D generation. We repurpose the VGGT reconstruction model to produce geometric latents by training an adapter on its tokens, which are regularized to align with the appearance latents of pre-trained video diffusion models. By jointly generating these disentangled yet aligned latents, Gen3R produces both RGB videos and corresponding 3D geometry, including camera poses, depth maps, and global point clouds. Experiments demonstrate that our approach achieves state-of-the-art results in single- and multi-image conditioned 3D scene generation. Additionally, our method can enhance the robustness of reconstruction by leveraging generative priors, demonstrating the mutual benefit of tightly coupling reconstruction and generative models.
695fa6f82450f142afb3c59b
Jiaxin Huang
claimed_verified
2026-01-09T08:35:46.425Z
63e367d3fae035bdc4c347fc
/avatars/239da0c9287bf965348aff4890e94722.svg
Jiaxin Huang
JaceyH919
user
JaceyH919
695fa6f82450f142afb3c59c
Yuanbo Yang
695fa6f82450f142afb3c59d
Bangbang Yang
695fa6f82450f142afb3c59e
Lin Ma
695fa6f82450f142afb3c59f
Yuewen Ma
695fa6f82450f142afb3c5a0
Yiyi Liao
Gen3R: 3D Scene Generation Meets Feed-Forward Reconstruction
We present Gen3R, a method that bridges the strong priors of foundational reconstruction models and video diffusion models for scene-level 3D generation. We repurpose the VGGT reconstruction model to produce geometric latents by training an adapter on its tokens, which are regularized to align with the appearance latents of pre-trained video diffusion models. By jointly generating these disentangled yet aligned latents, Gen3R produces both RGB videos and corresponding 3D geometry, including camera poses, depth maps, and global point clouds. Experiments demonstrate that our approach achieves state-of-the-art results in single- and multi-image conditioned 3D scene generation. Additionally, our method can enhance the robustness of reconstruction by leveraging generative priors, demonstrating the mutual benefit of tightly coupling reconstruction and generative models.
Gen3R combines foundational reconstruction models with video diffusion models to generate 3D scenes with RGB videos and geometric information through aligned latents.
VGGT reconstruction model video diffusion models geometric latents appearance latents adapter disentangled latents camera poses depth maps global point clouds