ShotVerse: Advancing Cinematic Camera Control for Text-Driven Multi-Shot Video Creation
Text-driven video generation has democratized film creation, but camera control in cinematic multi-shot scenarios remains a significant block. Implicit textual prompts lack precision, while explicit trajectory conditioning imposes prohibitive manual overhead and often triggers execution failures in current models. To overcome this bottleneck, we propose a data-centric paradigm shift, positing that aligned (Caption, Trajectory, Video) triplets form an inherent joint distribution that can connect automated plotting and precise execution. Guided by this insight, we present ShotVerse, a "Plan-then-Control" framework that decouples generation into two collaborative agents: a VLM (Vision-Language Model)-based Planner that leverages spatial priors to obtain cinematic, globally aligned trajectories from text, and a Controller that renders these trajectories into multi-shot video content via a camera adapter. Central to our approach is the construction of a data foundation: we design an automated multi-shot camera calibration pipeline aligns disjoint single-shot trajectories into a unified global coordinate system. This facilitates the curation of ShotVerse-Bench, a high-fidelity cinematic dataset with a three-track evaluation protocol that serves as the bedrock for our framework. Extensive experiments demonstrate that ShotVerse effectively bridges the gap between unreliable textual control and labor-intensive manual plotting, achieving superior cinematic aesthetics and generating multi-shot videos that are both camera-accurate and cross-shot consistent.
69b368712ad6c5d4cf48696a
Songlin Yang
admin_assigned
2026-03-13T08:57:44.363Z
66f64956ad4fe83c91776459
/avatars/2014de658ac12413754dcf70bc34333e.svg
Eddie
EddieYang428
user
EddieYang428
69b368712ad6c5d4cf48696b
Zhe Wang
69b368712ad6c5d4cf48696c
Xuyi Yang
69b368712ad6c5d4cf48696d
Songchun Zhang
69b368712ad6c5d4cf48696e
Xianghao Kong
claimed_verified
2026-03-16T10:43:54.741Z
6428fd124fe87caede856311
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/5OrNPwZkxu3Dm1IInCxML.jpeg
Xianghao Kong
refkxh
user
refkxh
69b368712ad6c5d4cf48696f
Taiyi Wu
69b368712ad6c5d4cf486970
Xiaotong Zhao
69b368712ad6c5d4cf486971
Ran Zhang
69b368712ad6c5d4cf486972
Alan Zhao
69b368712ad6c5d4cf486973
Anyi Rao
admin_assigned
2026-03-13T08:58:19.671Z
63f8130749569335b679af62
https://cdn-avatars.huggingface.co/v1/production/uploads/63f8130749569335b679af62/vgTu23-y0UKocwAGqNMwT.jpeg
Anyi Rao
anyirao
user
anyirao
ShotVerse: Advancing Cinematic Camera Control for Text-Driven Multi-Shot Video Creation
Text-driven video generation has democratized film creation, but camera control in cinematic multi-shot scenarios remains a significant block. Implicit textual prompts lack precision, while explicit trajectory conditioning imposes prohibitive manual overhead and often triggers execution failures in current models. To overcome this bottleneck, we propose a data-centric paradigm shift, positing that aligned (Caption, Trajectory, Video) triplets form an inherent joint distribution that can connect automated plotting and precise execution. Guided by this insight, we present ShotVerse, a "Plan-then-Control" framework that decouples generation into two collaborative agents: a VLM (Vision-Language Model)-based Planner that leverages spatial priors to obtain cinematic, globally aligned trajectories from text, and a Controller that renders these trajectories into multi-shot video content via a camera adapter. Central to our approach is the construction of a data foundation: we design an automated multi-shot camera calibration pipeline aligns disjoint single-shot trajectories into a unified global coordinate system. This facilitates the curation of ShotVerse-Bench, a high-fidelity cinematic dataset with a three-track evaluation protocol that serves as the bedrock for our framework. Extensive experiments demonstrate that ShotVerse effectively bridges the gap between unreliable textual control and labor-intensive manual plotting, achieving superior cinematic aesthetics and generating multi-shot videos that are both camera-accurate and cross-shot consistent.
ShotVerse enables cinematic video generation through a plan-then-control framework that decouples textual planning from camera control using vision-language models and trajectory alignment.
Vision-Language Model camera adapter multi-shot video generation cinematic aesthetics trajectory alignment global coordinate system ShotVerse-Bench cross-shot consistency