DualCamCtrl: Dual-Branch Diffusion Model for Geometry-Aware Camera-Controlled Video Generation
This paper presents DualCamCtrl, a novel end-to-end diffusion model for camera-controlled video generation. Recent works have advanced this field by representing camera poses as ray-based conditions, yet they often lack sufficient scene understanding and geometric awareness. DualCamCtrl specifically targets this limitation by introducing a dual-branch framework that mutually generates camera-consistent RGB and depth sequences. To harmonize these two modalities, we further propose the Semantic Guided Mutual Alignment (SIGMA) mechanism, which performs RGB-depth fusion in a semantics-guided and mutually reinforced manner. These designs collectively enable DualCamCtrl to better disentangle appearance and geometry modeling, generating videos that more faithfully adhere to the specified camera trajectories. Additionally, we analyze and reveal the distinct influence of depth and camera poses across denoising stages and further demonstrate that early and late stages play complementary roles in forming global structure and refining local details. Extensive experiments demonstrate that DualCamCtrl achieves more consistent camera-controlled video generation, with over 40\% reduction in camera motion errors compared with prior methods. Our project page: https://soyouthinkyoucantell.github.io/dualcamctrl-page/
692ea20837312eaa83fd8988
Hongfei Zhang
claimed_verified
2025-12-03T09:17:37.157Z
66864408c1d35ab079488662
https://cdn-avatars.huggingface.co/v1/production/uploads/66864408c1d35ab079488662/U-btVsJ65j7m6vFLVIka8.jpeg
Hongfei (Faye) Zhang
FayeHongfeiZhang
user
FayeHongfeiZhang
692ea20837312eaa83fd8989
Kanghao Chen
692ea20837312eaa83fd898a
Zixin Zhang
claimed_verified
2025-12-08T08:33:26.848Z
674aa9af9494dd9106006c27
/avatars/2f04bb009983ec0ade738aa7941cf6dc.svg
Zixin Zhang
zhangzixin02
user
zhangzixin02
692ea20837312eaa83fd898b
Harold Haodong Chen
claimed_verified
2025-12-03T09:16:15.369Z
6570450a78d7aca0c361a177
https://cdn-avatars.huggingface.co/v1/production/uploads/6570450a78d7aca0c361a177/MX7jHhTQwLs-BvYIu5rqb.jpeg
Harold Chen
Harold328
user
Harold328
692ea20837312eaa83fd898c
Yuanhuiyi Lyu
692ea20837312eaa83fd898d
Yuqi Zhang
claimed_verified
2025-12-03T09:16:13.161Z
692b0ee2cc35d0556d48e290
https://cdn-avatars.huggingface.co/v1/production/uploads/692b0ee2cc35d0556d48e290/rXH58_Ub7nywDlIDpUcph.png
YuqiZhang
murphy0103
user
murphy0103
692ea20837312eaa83fd898e
Shuai Yang
692ea20837312eaa83fd898f
Kun Zhou
692ea20837312eaa83fd8990
Yingcong Chen
DualCamCtrl: Dual-Branch Diffusion Model for Geometry-Aware Camera-Controlled Video Generation
This paper presents DualCamCtrl, a novel end-to-end diffusion model for camera-controlled video generation. Recent works have advanced this field by representing camera poses as ray-based conditions, yet they often lack sufficient scene understanding and geometric awareness. DualCamCtrl specifically targets this limitation by introducing a dual-branch framework that mutually generates camera-consistent RGB and depth sequences. To harmonize these two modalities, we further propose the Semantic Guided Mutual Alignment (SIGMA) mechanism, which performs RGB-depth fusion in a semantics-guided and mutually reinforced manner. These designs collectively enable DualCamCtrl to better disentangle appearance and geometry modeling, generating videos that more faithfully adhere to the specified camera trajectories. Additionally, we analyze and reveal the distinct influence of depth and camera poses across denoising stages and further demonstrate that early and late stages play complementary roles in forming global structure and refining local details. Extensive experiments demonstrate that DualCamCtrl achieves more consistent camera-controlled video generation, with over 40\% reduction in camera motion errors compared with prior methods. Our project page: https://soyouthinkyoucantell.github.io/dualcamctrl-page/
DualCamCtrl is a diffusion model for camera-controlled video generation that uses a dual-branch framework and Semantic Guided Mutual Alignment to improve consistency and disentangle appearance and geometry modeling.
diffusion model camera-controlled video generation ray-based conditions dual-branch framework Semantic Guided Mutual Alignment RGB depth sequences RGB-depth fusion appearance and geometry modeling denoising stages camera motion errors