Lyra 2.0: Explorable Generative 3D Worlds
Recent advances in video generation enable a new paradigm for 3D scene creation: generating camera-controlled videos that simulate scene walkthroughs, then lifting them to 3D via feed-forward reconstruction techniques. This generative reconstruction approach combines the visual fidelity and creative capacity of video models with 3D outputs ready for real-time rendering and simulation. Scaling to large, complex environments requires 3D-consistent video generation over long camera trajectories with large viewpoint changes and location revisits, a setting where current video models degrade quickly. Existing methods for long-horizon generation are fundamentally limited by two forms of degradation: spatial forgetting and temporal drifting. As exploration proceeds, previously observed regions fall outside the model's temporal context, forcing the model to hallucinate structures when revisited. Meanwhile, autoregressive generation accumulates small synthesis errors over time, gradually distorting scene appearance and geometry. We present Lyra 2.0, a framework for generating persistent, explorable 3D worlds at scale. To address spatial forgetting, we maintain per-frame 3D geometry and use it solely for information routing -- retrieving relevant past frames and establishing dense correspondences with the target viewpoints -- while relying on the generative prior for appearance synthesis. To address temporal drifting, we train with self-augmented histories that expose the model to its own degraded outputs, teaching it to correct drift rather than propagate it. Together, these enable substantially longer and 3D-consistent video trajectories, which we leverage to fine-tune feed-forward reconstruction models that reliably recover high-quality 3D scenes.
69df0082d3c84fcbf292180b
Tianchang Shen
69df0082d3c84fcbf292180c
Sherwin Bahmani
69df0082d3c84fcbf292180d
Kai He
69df0082d3c84fcbf292180e
Sangeetha Grama Srinivasan
69df0082d3c84fcbf292180f
Tianshi Cao
69df0082d3c84fcbf2921810
Jiawei Ren
69df0082d3c84fcbf2921811
Ruilong Li
69df0082d3c84fcbf2921812
Zian Wang
69df0082d3c84fcbf2921813
Nicholas Sharp
69df0082d3c84fcbf2921814
Zan Gojcic
69df0082d3c84fcbf2921815
Sanja Fidler
69df0082d3c84fcbf2921816
Jiahui Huang
69df0082d3c84fcbf2921817
Huan Ling
69df0082d3c84fcbf2921818
Jun Gao
69df0082d3c84fcbf2921819
Xuanchi Ren
Lyra 2.0: Explorable Generative 3D Worlds
Recent advances in video generation enable a new paradigm for 3D scene creation: generating camera-controlled videos that simulate scene walkthroughs, then lifting them to 3D via feed-forward reconstruction techniques. This generative reconstruction approach combines the visual fidelity and creative capacity of video models with 3D outputs ready for real-time rendering and simulation. Scaling to large, complex environments requires 3D-consistent video generation over long camera trajectories with large viewpoint changes and location revisits, a setting where current video models degrade quickly. Existing methods for long-horizon generation are fundamentally limited by two forms of degradation: spatial forgetting and temporal drifting. As exploration proceeds, previously observed regions fall outside the model's temporal context, forcing the model to hallucinate structures when revisited. Meanwhile, autoregressive generation accumulates small synthesis errors over time, gradually distorting scene appearance and geometry. We present Lyra 2.0, a framework for generating persistent, explorable 3D worlds at scale. To address spatial forgetting, we maintain per-frame 3D geometry and use it solely for information routing -- retrieving relevant past frames and establishing dense correspondences with the target viewpoints -- while relying on the generative prior for appearance synthesis. To address temporal drifting, we train with self-augmented histories that expose the model to its own degraded outputs, teaching it to correct drift rather than propagate it. Together, these enable substantially longer and 3D-consistent video trajectories, which we leverage to fine-tune feed-forward reconstruction models that reliably recover high-quality 3D scenes.
Lyra 2.0 enables large-scale 3D scene creation through persistent video generation that addresses spatial forgetting and temporal drifting issues in long-horizon video models.
video generation 3D scene creation camera-controlled videos feed-forward reconstruction spatial forgetting temporal drifting self-augmented histories 3D-consistent video trajectories generative prior dense correspondences