Tri-Prompting: Video Diffusion with Unified Control over Scene, Subject, and Motion
Recent video diffusion models have made remarkable strides in visual quality, yet precise, fine-grained control remains a key bottleneck that limits practical customizability for content creation. For AI video creators, three forms of control are crucial: (i) scene composition, (ii) multi-view consistent subject customization, and (iii) camera-pose or object-motion adjustment. Existing methods typically handle these dimensions in isolation, with limited support for multi-view subject synthesis and identity preservation under arbitrary pose changes. This lack of a unified architecture makes it difficult to support versatile, jointly controllable video. We introduce Tri-Prompting, a unified framework and two-stage training paradigm that integrates scene composition, multi-view subject consistency, and motion control. Our approach leverages a dual-condition motion module driven by 3D tracking points for background scenes and downsampled RGB cues for foreground subjects. To ensure a balance between controllability and visual realism, we further propose an inference ControlNet scale schedule. Tri-Prompting supports novel workflows, including 3D-aware subject insertion into any scenes and manipulation of existing subjects in an image. Experimental results demonstrate that Tri-Prompting significantly outperforms specialized baselines such as Phantom and DaS in multi-view subject identity, 3D consistency, and motion accuracy.
69b97f03ebc57f8925831f4b
Zhenghong Zhou
69b97f03ebc57f8925831f4c
Xiaohang Zhan
69b97f03ebc57f8925831f4d
Zhiqin Chen
69b97f03ebc57f8925831f4e
Soo Ye Kim
69b97f03ebc57f8925831f4f
Nanxuan Zhao
69b97f03ebc57f8925831f50
Haitian Zheng
69b97f03ebc57f8925831f51
Qing Liu
69b97f03ebc57f8925831f52
He Zhang
69b97f03ebc57f8925831f53
Zhe Lin
69b97f03ebc57f8925831f54
Yuqian Zhou
69b97f03ebc57f8925831f55
Jiebo Luo
Tri-Prompting: Video Diffusion with Unified Control over Scene, Subject, and Motion
Recent video diffusion models have made remarkable strides in visual quality, yet precise, fine-grained control remains a key bottleneck that limits practical customizability for content creation. For AI video creators, three forms of control are crucial: (i) scene composition, (ii) multi-view consistent subject customization, and (iii) camera-pose or object-motion adjustment. Existing methods typically handle these dimensions in isolation, with limited support for multi-view subject synthesis and identity preservation under arbitrary pose changes. This lack of a unified architecture makes it difficult to support versatile, jointly controllable video. We introduce Tri-Prompting, a unified framework and two-stage training paradigm that integrates scene composition, multi-view subject consistency, and motion control. Our approach leverages a dual-condition motion module driven by 3D tracking points for background scenes and downsampled RGB cues for foreground subjects. To ensure a balance between controllability and visual realism, we further propose an inference ControlNet scale schedule. Tri-Prompting supports novel workflows, including 3D-aware subject insertion into any scenes and manipulation of existing subjects in an image. Experimental results demonstrate that Tri-Prompting significantly outperforms specialized baselines such as Phantom and DaS in multi-view subject identity, 3D consistency, and motion accuracy.
Tri-Prompting presents a unified framework for video diffusion that enables joint control of scene composition, multi-view subject consistency, and motion, achieving superior performance in identity preservation and 3D consistency.
video diffusion models scene composition multi-view subject consistency camera-pose adjustment object-motion adjustment dual-condition motion module 3D tracking points downsampled RGB cues ControlNet scale schedule 3D-aware subject insertion