Stereo World Model: Camera-Guided Stereo Video Generation
We present StereoWorld, a camera-conditioned stereo world model that jointly learns appearance and binocular geometry for end-to-end stereo video generation.Unlike monocular RGB or RGBD approaches, StereoWorld operates exclusively within the RGB modality, while simultaneously grounding geometry directly from disparity. To efficiently achieve consistent stereo generation, our approach introduces two key designs: (1) a unified camera-frame RoPE that augments latent tokens with camera-aware rotary positional encoding, enabling relative, view- and time-consistent conditioning while preserving pretrained video priors via a stable attention initialization; and (2) a stereo-aware attention decomposition that factors full 4D attention into 3D intra-view attention plus horizontal row attention, leveraging the epipolar prior to capture disparity-aligned correspondences with substantially lower compute. Across benchmarks, StereoWorld improves stereo consistency, disparity accuracy, and camera-motion fidelity over strong monocular-then-convert pipelines, achieving more than 3x faster generation with an additional 5% gain in viewpoint consistency. Beyond benchmarks, StereoWorld enables end-to-end binocular VR rendering without depth estimation or inpainting, enhances embodied policy learning through metric-scale depth grounding, and is compatible with long-video distillation for extended interactive stereo synthesis.
69bb6e5fba2b62a5fb0b0c47
Yang-Tian Sun
69bb6e5fba2b62a5fb0b0c48
Zehuan Huang
69bb6e5fba2b62a5fb0b0c49
Yifan Niu
69bb6e5fba2b62a5fb0b0c4a
Lin Ma
69bb6e5fba2b62a5fb0b0c4b
Yan-Pei Cao
69bb6e5fba2b62a5fb0b0c4c
Yuewen Ma
69bb6e5fba2b62a5fb0b0c4d
Xiaojuan Qi
Stereo World Model: Camera-Guided Stereo Video Generation
We present StereoWorld, a camera-conditioned stereo world model that jointly learns appearance and binocular geometry for end-to-end stereo video generation.Unlike monocular RGB or RGBD approaches, StereoWorld operates exclusively within the RGB modality, while simultaneously grounding geometry directly from disparity. To efficiently achieve consistent stereo generation, our approach introduces two key designs: (1) a unified camera-frame RoPE that augments latent tokens with camera-aware rotary positional encoding, enabling relative, view- and time-consistent conditioning while preserving pretrained video priors via a stable attention initialization; and (2) a stereo-aware attention decomposition that factors full 4D attention into 3D intra-view attention plus horizontal row attention, leveraging the epipolar prior to capture disparity-aligned correspondences with substantially lower compute. Across benchmarks, StereoWorld improves stereo consistency, disparity accuracy, and camera-motion fidelity over strong monocular-then-convert pipelines, achieving more than 3x faster generation with an additional 5% gain in viewpoint consistency. Beyond benchmarks, StereoWorld enables end-to-end binocular VR rendering without depth estimation or inpainting, enhances embodied policy learning through metric-scale depth grounding, and is compatible with long-video distillation for extended interactive stereo synthesis.
StereoWorld is a camera-conditioned stereo world model that generates stereo videos end-to-end using RGB modality while maintaining geometric consistency and efficiency through novel attention mechanisms.
stereo world model camera-conditioned appearance binocular geometry stereo video generation latent tokens camera-aware rotary positional encoding RoPE stable attention initialization stereo-aware attention decomposition 4D attention 3D intra-view attention horizontal row attention epipolar prior disparity-aligned correspondences viewpoint consistency binocular VR rendering embodied policy learning metric-scale depth grounding long-video distillation