Repurposing Geometric Foundation Models for Multi-view Diffusion
While recent advances in generative latent spaces have driven substantial progress in single-image generation, the optimal latent space for novel view synthesis (NVS) remains largely unexplored. In particular, NVS requires geometrically consistent generation across viewpoints, but existing approaches typically operate in a view-independent VAE latent space. In this paper, we propose Geometric Latent Diffusion (GLD), a framework that repurposes the geometrically consistent feature space of geometric foundation models as the latent space for multi-view diffusion. We show that these features not only support high-fidelity RGB reconstruction but also encode strong cross-view geometric correspondences, providing a well-suited latent space for NVS. Our experiments demonstrate that GLD outperforms both VAE and RAE on 2D image quality and 3D consistency metrics, while accelerating training by more than 4.4x compared to the VAE latent space. Notably, GLD remains competitive with state-of-the-art methods that leverage large-scale text-to-image pretraining, despite training its diffusion model from scratch without such generative pretraining.
69c26b7a995349774064f573
Wooseok Jang
69c26b7a995349774064f574
Seonghu Jeon
69c26b7a995349774064f575
Jisang Han
claimed_verified
2026-03-24T13:20:45.849Z
64886f020a30bd0e7bd26dc3
/avatars/f02e7996a95968ba5cf9d5ba1de6d8b0.svg
Jisang Han
onground
user
onground
69c26b7a995349774064f576
Jinhyeok Choi
69c26b7a995349774064f577
Minkyung Kwon
69c26b7a995349774064f578
Seungryong Kim
69c26b7a995349774064f579
Saining Xie
69c26b7a995349774064f57a
Sainan Liu
Repurposing Geometric Foundation Models for Multi-view Diffusion
While recent advances in generative latent spaces have driven substantial progress in single-image generation, the optimal latent space for novel view synthesis (NVS) remains largely unexplored. In particular, NVS requires geometrically consistent generation across viewpoints, but existing approaches typically operate in a view-independent VAE latent space. In this paper, we propose Geometric Latent Diffusion (GLD), a framework that repurposes the geometrically consistent feature space of geometric foundation models as the latent space for multi-view diffusion. We show that these features not only support high-fidelity RGB reconstruction but also encode strong cross-view geometric correspondences, providing a well-suited latent space for NVS. Our experiments demonstrate that GLD outperforms both VAE and RAE on 2D image quality and 3D consistency metrics, while accelerating training by more than 4.4x compared to the VAE latent space. Notably, GLD remains competitive with state-of-the-art methods that leverage large-scale text-to-image pretraining, despite training its diffusion model from scratch without such generative pretraining.
Geometric Latent Diffusion (GLD) framework utilizes geometric foundation models' feature space as latent space for novel view synthesis, achieving superior 2D and 3D performance while reducing training time significantly.
generative latent spaces novel view synthesis VAE latent space geometric foundation models latent space multi-view diffusion cross-view geometric correspondences 2D image quality 3D consistency diffusion models training acceleration