4D-LRM: Large Space-Time Reconstruction Model From and To Any View at
  Any Time
Can we scale 4D pretraining to learn general space-time representations that
reconstruct an object from a few views at some times to any view at any time?
We provide an affirmative answer with 4D-LRM, the first large-scale 4D
reconstruction model that takes input from unconstrained views and timestamps
and renders arbitrary novel view-time combinations. Unlike prior 4D approaches,
e.g., optimization-based, geometry-based, or generative, that struggle with
efficiency, generalization, or faithfulness, 4D-LRM learns a unified space-time
representation and directly predicts per-pixel 4D Gaussian primitives from
posed image tokens across time, enabling fast, high-quality rendering at, in
principle, infinite frame rate. Our results demonstrate that scaling
spatiotemporal pretraining enables accurate and efficient 4D reconstruction. We
show that 4D-LRM generalizes to novel objects, interpolates across time, and
handles diverse camera setups. It reconstructs 24-frame sequences in one
forward pass with less than 1.5 seconds on a single A100 GPU.
685b168dd2ee4fac76521d68
Ziqiao Ma
claimed_verified
2025-06-25T08:09:00.720Z
630cfc45b66f088d547b2768
https://cdn-avatars.huggingface.co/v1/production/uploads/630cfc45b66f088d547b2768/9-dMts2xFVbPmPHJGGqBx.png
Martin Ziqiao Ma
marstin
user
marstin
685b168dd2ee4fac76521d69
Xuweiyi Chen
685b168dd2ee4fac76521d6a
Shoubin Yu
685b168dd2ee4fac76521d6b
Sai Bi
685b168dd2ee4fac76521d6c
Kai Zhang
685b168dd2ee4fac76521d6d
Chen Ziwen
685b168dd2ee4fac76521d6e
Sihan Xu
claimed_verified
2025-12-19T09:09:54.641Z
63f233820a16587ea967adc2
https://cdn-avatars.huggingface.co/v1/production/uploads/63f233820a16587ea967adc2/1nSoZofPV7UseXzjI2qAH.png
Sihan XU
sihanxu
user
sihanxu
685b168dd2ee4fac76521d6f
Jianing Yang
685b168dd2ee4fac76521d70
Zexiang Xu
685b168dd2ee4fac76521d71
Kalyan Sunkavalli
685b168dd2ee4fac76521d72
Mohit Bansal
685b168dd2ee4fac76521d73
Joyce Chai
685b168dd2ee4fac76521d74
Hao Tan
4D-LRM: Large Space-Time Reconstruction Model From and To Any View at
  Any Time
Can we scale 4D pretraining to learn general space-time representations that
reconstruct an object from a few views at some times to any view at any time?
We provide an affirmative answer with 4D-LRM, the first large-scale 4D
reconstruction model that takes input from unconstrained views and timestamps
and renders arbitrary novel view-time combinations. Unlike prior 4D approaches,
e.g., optimization-based, geometry-based, or generative, that struggle with
efficiency, generalization, or faithfulness, 4D-LRM learns a unified space-time
representation and directly predicts per-pixel 4D Gaussian primitives from
posed image tokens across time, enabling fast, high-quality rendering at, in
principle, infinite frame rate. Our results demonstrate that scaling
spatiotemporal pretraining enables accurate and efficient 4D reconstruction. We
show that 4D-LRM generalizes to novel objects, interpolates across time, and
handles diverse camera setups. It reconstructs 24-frame sequences in one
forward pass with less than 1.5 seconds on a single A100 GPU.
4D-LRM is a large-scale model that efficiently reconstructs objects from multiple views and times into any view-time combination using space-time representations and Gaussian primitives.
4D-LRM space-time representation Gaussian primitives posed image tokens 4D reconstruction frame rate