Out of Sight but Not Out of Mind: Hybrid Memory for Dynamic Video World Models
Video world models have shown immense potential in simulating the physical world, yet existing memory mechanisms primarily treat environments as static canvases. When dynamic subjects hide out of sight and later re-emerge, current methods often struggle, leading to frozen, distorted, or vanishing subjects. To address this, we introduce Hybrid Memory, a novel paradigm requiring models to simultaneously act as precise archivists for static backgrounds and vigilant trackers for dynamic subjects, ensuring motion continuity during out-of-view intervals. To facilitate research in this direction, we construct HM-World, the first large-scale video dataset dedicated to hybrid memory. It features 59K high-fidelity clips with decoupled camera and subject trajectories, encompassing 17 diverse scenes, 49 distinct subjects, and meticulously designed exit-entry events to rigorously evaluate hybrid coherence. Furthermore, we propose HyDRA, a specialized memory architecture that compresses memory into tokens and utilizes a spatiotemporal relevance-driven retrieval mechanism. By selectively attending to relevant motion cues, HyDRA effectively preserves the identity and motion of hidden subjects. Extensive experiments on HM-World demonstrate that our method significantly outperforms state-of-the-art approaches in both dynamic subject consistency and overall generation quality.
69c687028508d7e37629b4bb
Kaijin Chen
admin_assigned
2026-03-30T10:17:44.832Z
6617739a0795b7cd10ad8f75
/avatars/87b24f8eb0673529b0083e1c61462484.svg
Kaijin Chen
Medivan
user
Medivan
69c687028508d7e37629b4bc
Dingkang Liang
admin_assigned
2026-03-30T10:17:37.170Z
67467b5979406f42a14517e9
https://cdn-avatars.huggingface.co/v1/production/uploads/67467b5979406f42a14517e9/wgnUxTd8vWOo0Cr2gaoyG.jpeg
Dingkang Liang
dkliang
user
dkliang
69c687028508d7e37629b4bd
Xin Zhou
claimed_verified
2026-03-30T10:24:10.071Z
6717c5c36bc2876059ed23ab
/avatars/52c68fb315760df5ef9323cd8ada5a3c.svg
Xin Zhou
LMD0311
user
LMD0311
69c687028508d7e37629b4be
Yikang Ding
admin_assigned
2026-03-30T10:17:57.347Z
66cf19c76116c7f53e909d49
/avatars/13d5d139c33b9e9bb1b251f2b0a7cd91.svg
dingyikang
dingyikang
user
dingyikang
69c687028508d7e37629b4bf
Xiaoqiang Liu
69c687028508d7e37629b4c0
Pengfei Wan
69c687028508d7e37629b4c1
Xiang Bai
Out of Sight but Not Out of Mind: Hybrid Memory for Dynamic Video World Models
Video world models have shown immense potential in simulating the physical world, yet existing memory mechanisms primarily treat environments as static canvases. When dynamic subjects hide out of sight and later re-emerge, current methods often struggle, leading to frozen, distorted, or vanishing subjects. To address this, we introduce Hybrid Memory, a novel paradigm requiring models to simultaneously act as precise archivists for static backgrounds and vigilant trackers for dynamic subjects, ensuring motion continuity during out-of-view intervals. To facilitate research in this direction, we construct HM-World, the first large-scale video dataset dedicated to hybrid memory. It features 59K high-fidelity clips with decoupled camera and subject trajectories, encompassing 17 diverse scenes, 49 distinct subjects, and meticulously designed exit-entry events to rigorously evaluate hybrid coherence. Furthermore, we propose HyDRA, a specialized memory architecture that compresses memory into tokens and utilizes a spatiotemporal relevance-driven retrieval mechanism. By selectively attending to relevant motion cues, HyDRA effectively preserves the identity and motion of hidden subjects. Extensive experiments on HM-World demonstrate that our method significantly outperforms state-of-the-art approaches in both dynamic subject consistency and overall generation quality.
Hybrid Memory enables video world models to maintain consistent tracking of dynamic subjects during occlusion by combining archival storage for static backgrounds with active tracking for moving objects, using a specialized architecture with tokenized memory and spatiotemporal retrieval mechanisms.
video world models hybrid memory dynamic subjects static backgrounds spatiotemporal relevance-driven retrieval tokenized memory motion continuity occlusion handling memory architecture generative modeling