BulletTime: Decoupled Control of Time and Camera Pose for Video Generation
Emerging video diffusion models achieve high visual fidelity but fundamentally couple scene dynamics with camera motion, limiting their ability to provide precise spatial and temporal control. We introduce a 4D-controllable video diffusion framework that explicitly decouples scene dynamics from camera pose, enabling fine-grained manipulation of both scene dynamics and camera viewpoint. Our framework takes continuous world-time sequences and camera trajectories as conditioning inputs, injecting them into the video diffusion model through a 4D positional encoding in the attention layer and adaptive normalizations for feature modulation. To train this model, we curate a unique dataset in which temporal and camera variations are independently parameterized; this dataset will be made public. Experiments show that our model achieves robust real-world 4D control across diverse timing patterns and camera trajectories, while preserving high generation quality and outperforming prior work in controllability. See our website for video results: https://19reborn.github.io/Bullet4D/
693254586d1060ca587a2753
Yiming Wang
693254586d1060ca587a2754
Qihang Zhang
693254586d1060ca587a2755
Shengqu Cai
693254586d1060ca587a2756
Tong Wu
693254586d1060ca587a2757
Jan Ackermann
693254586d1060ca587a2758
Zhengfei Kuang
693254586d1060ca587a2759
Yang Zheng
693254586d1060ca587a275a
Frano Rajič
693254586d1060ca587a275b
Siyu Tang
693254586d1060ca587a275c
Gordon Wetzstein
BulletTime: Decoupled Control of Time and Camera Pose for Video Generation
Emerging video diffusion models achieve high visual fidelity but fundamentally couple scene dynamics with camera motion, limiting their ability to provide precise spatial and temporal control. We introduce a 4D-controllable video diffusion framework that explicitly decouples scene dynamics from camera pose, enabling fine-grained manipulation of both scene dynamics and camera viewpoint. Our framework takes continuous world-time sequences and camera trajectories as conditioning inputs, injecting them into the video diffusion model through a 4D positional encoding in the attention layer and adaptive normalizations for feature modulation. To train this model, we curate a unique dataset in which temporal and camera variations are independently parameterized; this dataset will be made public. Experiments show that our model achieves robust real-world 4D control across diverse timing patterns and camera trajectories, while preserving high generation quality and outperforming prior work in controllability. See our website for video results: https://19reborn.github.io/Bullet4D/
A 4D-controllable video diffusion framework decouples scene dynamics from camera pose, enabling precise manipulation of both and achieving high-quality generation across diverse timing patterns and camera trajectories.
video diffusion models scene dynamics camera pose 4D positional encoding attention layer adaptive normalizations feature modulation 4D control