Video4Spatial: Towards Visuospatial Intelligence with Context-Guided Video Generation
We investigate whether video generative models can exhibit visuospatial intelligence, a capability central to human cognition, using only visual data. To this end, we present Video4Spatial, a framework showing that video diffusion models conditioned solely on video-based scene context can perform complex spatial tasks. We validate on two tasks: scene navigation - following camera-pose instructions while remaining consistent with 3D geometry of the scene, and object grounding - which requires semantic localization, instruction following, and planning. Both tasks use video-only inputs, without auxiliary modalities such as depth or poses. With simple yet effective design choices in the framework and data curation, Video4Spatial demonstrates strong spatial understanding from video context: it plans navigation and grounds target objects end-to-end, follows camera-pose instructions while maintaining spatial consistency, and generalizes to long contexts and out-of-domain environments. Taken together, these results advance video generative models toward general visuospatial reasoning.
692fb17b26742347f61dac7e
Zeqi Xiao
692fb17b26742347f61dac7f
Yiwei Zhao
692fb17b26742347f61dac80
Lingxiao Li
692fb17b26742347f61dac81
Yushi Lan
692fb17b26742347f61dac82
Yu Ning
692fb17b26742347f61dac83
Rahul Garg
692fb17b26742347f61dac84
Roshni Cooper
692fb17b26742347f61dac85
Mohammad H. Taghavi
692fb17b26742347f61dac86
Xingang Pan
Video4Spatial: Towards Visuospatial Intelligence with Context-Guided Video Generation
We investigate whether video generative models can exhibit visuospatial intelligence, a capability central to human cognition, using only visual data. To this end, we present Video4Spatial, a framework showing that video diffusion models conditioned solely on video-based scene context can perform complex spatial tasks. We validate on two tasks: scene navigation - following camera-pose instructions while remaining consistent with 3D geometry of the scene, and object grounding - which requires semantic localization, instruction following, and planning. Both tasks use video-only inputs, without auxiliary modalities such as depth or poses. With simple yet effective design choices in the framework and data curation, Video4Spatial demonstrates strong spatial understanding from video context: it plans navigation and grounds target objects end-to-end, follows camera-pose instructions while maintaining spatial consistency, and generalizes to long contexts and out-of-domain environments. Taken together, these results advance video generative models toward general visuospatial reasoning.
Video4Spatial demonstrates that video diffusion models can perform complex spatial tasks using only visual data, achieving strong spatial understanding and generalization.
video diffusion models scene navigation object grounding semantic localization instruction following planning 3D geometry video-only inputs spatial consistency generalization