LagerNVS: Latent Geometry for Fully Neural Real-time Novel View Synthesis
Recent work has shown that neural networks can perform 3D tasks such as Novel View Synthesis (NVS) without explicit 3D reconstruction. Even so, we argue that strong 3D inductive biases are still helpful in the design of such networks. We show this point by introducing LagerNVS, an encoder-decoder neural network for NVS that builds on `3D-aware' latent features. The encoder is initialized from a 3D reconstruction network pre-trained using explicit 3D supervision. This is paired with a lightweight decoder, and trained end-to-end with photometric losses. LagerNVS achieves state-of-the-art deterministic feed-forward Novel View Synthesis (including 31.4 PSNR on Re10k), with and without known cameras, renders in real time, generalizes to in-the-wild data, and can be paired with a diffusion decoder for generative extrapolation.
69c161fde9374d7b3df94dc2
Stanislaw Szymanowicz
claimed_verified
2026-03-24T09:22:47.947Z
65fc155083e0b6395eab102a
/avatars/1b9e6fa376d192c0c85ff2f82ff89008.svg
Stan Szymanowicz
szymanowiczs
user
szymanowiczs
69c161fde9374d7b3df94dc3
Minghao Chen
69c161fde9374d7b3df94dc4
Jianyuan Wang
69c161fde9374d7b3df94dc5
Christian Rupprecht
69c161fde9374d7b3df94dc6
Andrea Vedaldi
LagerNVS: Latent Geometry for Fully Neural Real-time Novel View Synthesis
Recent work has shown that neural networks can perform 3D tasks such as Novel View Synthesis (NVS) without explicit 3D reconstruction. Even so, we argue that strong 3D inductive biases are still helpful in the design of such networks. We show this point by introducing LagerNVS, an encoder-decoder neural network for NVS that builds on `3D-aware' latent features. The encoder is initialized from a 3D reconstruction network pre-trained using explicit 3D supervision. This is paired with a lightweight decoder, and trained end-to-end with photometric losses. LagerNVS achieves state-of-the-art deterministic feed-forward Novel View Synthesis (including 31.4 PSNR on Re10k), with and without known cameras, renders in real time, generalizes to in-the-wild data, and can be paired with a diffusion decoder for generative extrapolation.
Neural networks can perform 3D tasks like novel view synthesis without explicit 3D reconstruction, but incorporating 3D inductive biases through encoder-decoder architectures with pre-trained 3D reconstruction networks improves performance and enables real-time rendering.
Novel View Synthesis 3D-aware latent features encoder-decoder neural network 3D reconstruction network photometric losses deterministic feed-forward real-time rendering generative extrapolation