MosaicMem: Hybrid Spatial Memory for Controllable Video World Models
Video diffusion models are moving beyond short, plausible clips toward world simulators that must remain consistent under camera motion, revisits, and intervention. Yet spatial memory remains a key bottleneck: explicit 3D structures can improve reprojection-based consistency but struggle to depict moving objects, while implicit memory often produces inaccurate camera motion even with correct poses. We propose Mosaic Memory (MosaicMem), a hybrid spatial memory that lifts patches into 3D for reliable localization and targeted retrieval, while exploiting the model's native conditioning to preserve prompt-following generation. MosaicMem composes spatially aligned patches in the queried view via a patch-and-compose interface, preserving what should persist while allowing the model to inpaint what should evolve. With PRoPE camera conditioning and two new memory alignment methods, experiments show improved pose adherence compared to implicit memory and stronger dynamic modeling than explicit baselines. MosaicMem further enables minute-level navigation, memory-based scene editing, and autoregressive rollout.
69bb693bba2b62a5fb0b0c25
Wei Yu
claimed_verified
2026-03-19T12:57:25.705Z
6350884a0f376d3c482bda54
/avatars/824c49aa2fb2e85801c001e2843c0576.svg
Wei Yu
yuweiao
user
yuweiao
69bb693bba2b62a5fb0b0c26
Runjia Qian
claimed_verified
2026-03-23T10:56:41.496Z
697b04bf95c7d6cade79b835
https://cdn-avatars.huggingface.co/v1/production/uploads/697b04bf95c7d6cade79b835/5urnq1O8o2Q5vgFIL1HTV.jpeg
Runjia Qian
rickchains
user
rickchains
69bb693bba2b62a5fb0b0c27
Yumeng Li
claimed_verified
2026-03-19T12:57:19.837Z
62a8e65cbad2c679dc0b2866
/avatars/08f06947880f74145005f0da1a82dfed.svg
Yumeng Li
Yumeng
user
Yumeng
69bb693bba2b62a5fb0b0c28
Liquan Wang
69bb693bba2b62a5fb0b0c29
Songheng Yin
69bb693bba2b62a5fb0b0c2a
Sri Siddarth Chakaravarthy P
69bb693bba2b62a5fb0b0c2b
Dennis Anthony
69bb693bba2b62a5fb0b0c2c
Yang Ye
69bb693bba2b62a5fb0b0c2d
Yidi Li
claimed_verified
2026-03-19T12:57:22.366Z
69bb748e8e2239e903af8b7e
/avatars/c58e3ec276e0830055b65ac21039c1b4.svg
yidiiii
yidiiii
user
yidiiii
69bb693bba2b62a5fb0b0c2e
Weiwei Wan
69bb693bba2b62a5fb0b0c2f
Animesh Garg
claimed_verified
2026-03-23T10:56:39.118Z
624a0cdfd9520873d3a11170
https://cdn-avatars.huggingface.co/v1/production/uploads/624a0cdfd9520873d3a11170/DEbChl9ar7C0HQG8Aej9Z.jpeg
Animesh Garg
animesh-garg
user
animesh-garg
MosaicMem: Hybrid Spatial Memory for Controllable Video World Models
Video diffusion models are moving beyond short, plausible clips toward world simulators that must remain consistent under camera motion, revisits, and intervention. Yet spatial memory remains a key bottleneck: explicit 3D structures can improve reprojection-based consistency but struggle to depict moving objects, while implicit memory often produces inaccurate camera motion even with correct poses. We propose Mosaic Memory (MosaicMem), a hybrid spatial memory that lifts patches into 3D for reliable localization and targeted retrieval, while exploiting the model's native conditioning to preserve prompt-following generation. MosaicMem composes spatially aligned patches in the queried view via a patch-and-compose interface, preserving what should persist while allowing the model to inpaint what should evolve. With PRoPE camera conditioning and two new memory alignment methods, experiments show improved pose adherence compared to implicit memory and stronger dynamic modeling than explicit baselines. MosaicMem further enables minute-level navigation, memory-based scene editing, and autoregressive rollout.
Video diffusion models use hybrid spatial memory to maintain consistency under camera motion and enable long-term scene editing and navigation.
video diffusion models spatial memory 3D structures implicit memory patch-and-compose interface PRoPE camera conditioning memory alignment methods autoregressive rollout