VerseCrafter: Dynamic Realistic Video World Model with 4D Geometric Control
Video world models aim to simulate dynamic, real-world environments, yet existing methods struggle to provide unified and precise control over camera and multi-object motion, as videos inherently operate dynamics in the projected 2D image plane. To bridge this gap, we introduce VerseCrafter, a 4D-aware video world model that enables explicit and coherent control over both camera and object dynamics within a unified 4D geometric world state. Our approach is centered on a novel 4D Geometric Control representation, which encodes the world state through a static background point cloud and per-object 3D Gaussian trajectories. This representation captures not only an object's path but also its probabilistic 3D occupancy over time, offering a flexible, category-agnostic alternative to rigid bounding boxes or parametric models. These 4D controls are rendered into conditioning signals for a pretrained video diffusion model, enabling the generation of high-fidelity, view-consistent videos that precisely adhere to the specified dynamics. Unfortunately, another major challenge lies in the scarcity of large-scale training data with explicit 4D annotations. We address this by developing an automatic data engine that extracts the required 4D controls from in-the-wild videos, allowing us to train our model on a massive and diverse dataset.
69606cd45b7998385e639489
Sixiao Zheng
claimed_verified
2026-01-09T08:35:19.760Z
6528fe394513680346a500ed
/avatars/3021a3be7d900e3928cdc375e2a2bf09.svg
Sixiao Zheng
sxzheng
user
sxzheng
69606cd45b7998385e63948a
Minghao Yin
69606cd45b7998385e63948b
Wenbo Hu
claimed_verified
2026-01-12T10:36:21.392Z
657a7458afbb0117ba15c59f
https://cdn-avatars.huggingface.co/v1/production/uploads/657a7458afbb0117ba15c59f/8_iwTS1UG_mKnfylFbLsY.jpeg
Wenbo Hu
wbhu-tc
user
wbhu-tc
69606cd45b7998385e63948c
Xiaoyu Li
69606cd45b7998385e63948d
Ying Shan
69606cd45b7998385e63948e
Yanwei Fu
admin_assigned
2026-01-09T15:57:43.763Z
6409fcc8f3dabf93824c84c6
/avatars/dd8fd579630e50ba3058c3829604478e.svg
YANWEI
yanweifuture
user
yanweifuture
VerseCrafter: Dynamic Realistic Video World Model with 4D Geometric Control
Video world models aim to simulate dynamic, real-world environments, yet existing methods struggle to provide unified and precise control over camera and multi-object motion, as videos inherently operate dynamics in the projected 2D image plane. To bridge this gap, we introduce VerseCrafter, a 4D-aware video world model that enables explicit and coherent control over both camera and object dynamics within a unified 4D geometric world state. Our approach is centered on a novel 4D Geometric Control representation, which encodes the world state through a static background point cloud and per-object 3D Gaussian trajectories. This representation captures not only an object's path but also its probabilistic 3D occupancy over time, offering a flexible, category-agnostic alternative to rigid bounding boxes or parametric models. These 4D controls are rendered into conditioning signals for a pretrained video diffusion model, enabling the generation of high-fidelity, view-consistent videos that precisely adhere to the specified dynamics. Unfortunately, another major challenge lies in the scarcity of large-scale training data with explicit 4D annotations. We address this by developing an automatic data engine that extracts the required 4D controls from in-the-wild videos, allowing us to train our model on a massive and diverse dataset.
VerseCrafter is a 4D-aware video world model that enables unified control over camera and object dynamics through 4D geometric control representation and video diffusion models.
video world models 4D geometric control point cloud 3D Gaussian trajectories video diffusion model view-consistent videos automatic data engine in-the-wild videos