ShotStream: Streaming Multi-Shot Video Generation for Interactive Storytelling
Multi-shot video generation is crucial for long narrative storytelling, yet current bidirectional architectures suffer from limited interactivity and high latency. We propose ShotStream, a novel causal multi-shot architecture that enables interactive storytelling and efficient on-the-fly frame generation. By reformulating the task as next-shot generation conditioned on historical context, ShotStream allows users to dynamically instruct ongoing narratives via streaming prompts. We achieve this by first fine-tuning a text-to-video model into a bidirectional next-shot generator, which is then distilled into a causal student via Distribution Matching Distillation. To overcome the challenges of inter-shot consistency and error accumulation inherent in autoregressive generation, we introduce two key innovations. First, a dual-cache memory mechanism preserves visual coherence: a global context cache retains conditional frames for inter-shot consistency, while a local context cache holds generated frames within the current shot for intra-shot consistency. And a RoPE discontinuity indicator is employed to explicitly distinguish the two caches to eliminate ambiguity. Second, to mitigate error accumulation, we propose a two-stage distillation strategy. This begins with intra-shot self-forcing conditioned on ground-truth historical shots and progressively extends to inter-shot self-forcing using self-generated histories, effectively bridging the train-test gap. Extensive experiments demonstrate that ShotStream generates coherent multi-shot videos with sub-second latency, achieving 16 FPS on a single GPU. It matches or exceeds the quality of slower bidirectional models, paving the way for real-time interactive storytelling. Training and inference code, as well as the models, are available on our
69c658e12de2ee20094edef3
Yawen Luo
admin_assigned
2026-03-30T10:18:41.356Z
66743477ab975c859114d410
/avatars/ac692cc336e383fb2cb53db6d1e3fe8c.svg
yawenluo
yawenluo
user
yawenluo
69c658e12de2ee20094edef4
Xiaoyu Shi
69c658e12de2ee20094edef5
Junhao Zhuang
admin_assigned
2026-03-30T10:18:58.978Z
64970d3d9c3b29dca8633f87
/avatars/11e3c9c66d28490d6d09925f9aa47cd1.svg
JunhaoZhuang
JunhaoZhuang
user
JunhaoZhuang
69c658e12de2ee20094edef6
Yutian Chen
69c658e12de2ee20094edef7
Quande Liu
admin_assigned
2026-03-30T10:19:18.067Z
692deb9b2ab84c5b653fb191
/avatars/0c8a60aa5d827cc5d305e84850ad0a47.svg
quandeliu
quandeliu
user
quandeliu
69c658e12de2ee20094edef8
Xintao Wang
admin_assigned
2026-03-30T10:19:25.021Z
60e272ca6c78a8c122b12127
https://cdn-avatars.huggingface.co/v1/production/uploads/60e272ca6c78a8c122b12127/xldEGBzGrU-bX6IwAw0Ie.jpeg
Xintao Wang
Xintao
user
Xintao
69c658e12de2ee20094edef9
Pengfei Wan
69c658e12de2ee20094edefa
Tianfan Xue
admin_assigned
2026-03-30T10:19:34.566Z
67631f57abfbd60470d4b3c3
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/5VrG0IZjYrDLiatOr4y06.png
Tianfan Xue
littlemouse9
user
littlemouse9
ShotStream: Streaming Multi-Shot Video Generation for Interactive Storytelling
Multi-shot video generation is crucial for long narrative storytelling, yet current bidirectional architectures suffer from limited interactivity and high latency. We propose ShotStream, a novel causal multi-shot architecture that enables interactive storytelling and efficient on-the-fly frame generation. By reformulating the task as next-shot generation conditioned on historical context, ShotStream allows users to dynamically instruct ongoing narratives via streaming prompts. We achieve this by first fine-tuning a text-to-video model into a bidirectional next-shot generator, which is then distilled into a causal student via Distribution Matching Distillation. To overcome the challenges of inter-shot consistency and error accumulation inherent in autoregressive generation, we introduce two key innovations. First, a dual-cache memory mechanism preserves visual coherence: a global context cache retains conditional frames for inter-shot consistency, while a local context cache holds generated frames within the current shot for intra-shot consistency. And a RoPE discontinuity indicator is employed to explicitly distinguish the two caches to eliminate ambiguity. Second, to mitigate error accumulation, we propose a two-stage distillation strategy. This begins with intra-shot self-forcing conditioned on ground-truth historical shots and progressively extends to inter-shot self-forcing using self-generated histories, effectively bridging the train-test gap. Extensive experiments demonstrate that ShotStream generates coherent multi-shot videos with sub-second latency, achieving 16 FPS on a single GPU. It matches or exceeds the quality of slower bidirectional models, paving the way for real-time interactive storytelling. Training and inference code, as well as the models, are available on our
ShotStream enables real-time interactive multi-shot video generation through causal architecture design, dual-cache memory mechanisms, and two-stage distillation to maintain visual consistency and reduce latency.
causal architecture multi-shot video generation next-shot generation text-to-video model distribution matching distillation dual-cache memory mechanism global context cache local context cache RoPE discontinuity indicator self-forcing two-stage distillation autoregressive generation inter-shot consistency intra-shot consistency error accumulation train-test gap