LiveWorld: Simulating Out-of-Sight Dynamics in Generative Video World Models
Recent generative video world models aim to simulate visual environment evolution, allowing an observer to interactively explore the scene via camera control. However, they implicitly assume that the world only evolves within the observer's field of view. Once an object leaves the observer's view, its state is "frozen" in memory, and revisiting the same region later often fails to reflect events that should have occurred in the meantime. In this work, we identify and formalize this overlooked limitation as the "out-of-sight dynamics" problem, which impedes video world models from representing a continuously evolving world. To address this issue, we propose LiveWorld, a novel framework that extends video world models to support persistent world evolution. Instead of treating the world as static observational memory, LiveWorld models a persistent global state composed of a static 3D background and dynamic entities that continue evolving even when unobserved. To maintain these unseen dynamics, LiveWorld introduces a monitor-based mechanism that autonomously simulates the temporal progression of active entities and synchronizes their evolved states upon revisiting, ensuring spatially coherent rendering. For evaluation, we further introduce LiveBench, a dedicated benchmark for the task of maintaining out-of-sight dynamics. Extensive experiments show that LiveWorld enables persistent event evolution and long-term scene consistency, bridging the gap between existing 2D observation-based memory and true 4D dynamic world simulation. The baseline and benchmark will be publicly available at https://zichengduan.github.io/LiveWorld/index.html.
69affbff9dbab2a62ec28066
Zicheng Duan
claimed_verified
2026-03-11T08:45:09.785Z
64e4446756b920ef00f2031c
/avatars/86305d1eac3917bf9859ac0ddbb4e845.svg
Zicheng Duan
ZichengD
user
ZichengD
69affbff9dbab2a62ec28067
Jiatong Xia
69affbff9dbab2a62ec28068
Zeyu Zhang
69affbff9dbab2a62ec28069
Wenbo Zhang
69affbff9dbab2a62ec2806a
Gengze Zhou
69affbff9dbab2a62ec2806b
Chenhui Gou
69affbff9dbab2a62ec2806c
Yefei He
69affbff9dbab2a62ec2806d
Feng Chen
69affbff9dbab2a62ec2806e
Xinyu Zhang
69affbff9dbab2a62ec2806f
Lingqiao Liu
LiveWorld: Simulating Out-of-Sight Dynamics in Generative Video World Models
Recent generative video world models aim to simulate visual environment evolution, allowing an observer to interactively explore the scene via camera control. However, they implicitly assume that the world only evolves within the observer's field of view. Once an object leaves the observer's view, its state is "frozen" in memory, and revisiting the same region later often fails to reflect events that should have occurred in the meantime. In this work, we identify and formalize this overlooked limitation as the "out-of-sight dynamics" problem, which impedes video world models from representing a continuously evolving world. To address this issue, we propose LiveWorld, a novel framework that extends video world models to support persistent world evolution. Instead of treating the world as static observational memory, LiveWorld models a persistent global state composed of a static 3D background and dynamic entities that continue evolving even when unobserved. To maintain these unseen dynamics, LiveWorld introduces a monitor-based mechanism that autonomously simulates the temporal progression of active entities and synchronizes their evolved states upon revisiting, ensuring spatially coherent rendering. For evaluation, we further introduce LiveBench, a dedicated benchmark for the task of maintaining out-of-sight dynamics. Extensive experiments show that LiveWorld enables persistent event evolution and long-term scene consistency, bridging the gap between existing 2D observation-based memory and true 4D dynamic world simulation. The baseline and benchmark will be publicly available at https://zichengduan.github.io/LiveWorld/index.html.
LiveWorld addresses the out-of-sight dynamics problem in video world models by introducing a persistent global state representation that maintains continuous evolution of dynamic entities beyond the observer's field of view.
video world models out-of-sight dynamics persistent world evolution 3D background dynamic entities monitor-based mechanism temporal progression spatial coherence 4D dynamic world simulation