StoryMem: Multi-shot Long Video Storytelling with Memory
Visual storytelling requires generating multi-shot videos with cinematic quality and long-range consistency. Inspired by human memory, we propose StoryMem, a paradigm that reformulates long-form video storytelling as iterative shot synthesis conditioned on explicit visual memory, transforming pre-trained single-shot video diffusion models into multi-shot storytellers. This is achieved by a novel Memory-to-Video (M2V) design, which maintains a compact and dynamically updated memory bank of keyframes from historical generated shots. The stored memory is then injected into single-shot video diffusion models via latent concatenation and negative RoPE shifts with only LoRA fine-tuning. A semantic keyframe selection strategy, together with aesthetic preference filtering, further ensures informative and stable memory throughout generation. Moreover, the proposed framework naturally accommodates smooth shot transitions and customized story generation applications. To facilitate evaluation, we introduce ST-Bench, a diverse benchmark for multi-shot video storytelling. Extensive experiments demonstrate that StoryMem achieves superior cross-shot consistency over previous methods while preserving high aesthetic quality and prompt adherence, marking a significant step toward coherent minute-long video storytelling.
694a1285335742716e932295
Kaiwen Zhang
claimed_verified
2025-12-25T20:48:23.344Z
641087ddf52d7eb22e0673dd
/avatars/946f29569aeda2cbd9ef2b94b1ab1391.svg
Kevin Zhang
Kevin-thu
user
Kevin-thu
694a1285335742716e932296
Liming Jiang
694a1285335742716e932297
Angtian Wang
694a1285335742716e932298
Jacob Zhiyuan Fang
694a1285335742716e932299
Tiancheng Zhi
694a1285335742716e93229a
Qing Yan
694a1285335742716e93229b
Hao Kang
694a1285335742716e93229c
Xin Lu
694a1285335742716e93229d
Xingang Pan
StoryMem: Multi-shot Long Video Storytelling with Memory
Visual storytelling requires generating multi-shot videos with cinematic quality and long-range consistency. Inspired by human memory, we propose StoryMem, a paradigm that reformulates long-form video storytelling as iterative shot synthesis conditioned on explicit visual memory, transforming pre-trained single-shot video diffusion models into multi-shot storytellers. This is achieved by a novel Memory-to-Video (M2V) design, which maintains a compact and dynamically updated memory bank of keyframes from historical generated shots. The stored memory is then injected into single-shot video diffusion models via latent concatenation and negative RoPE shifts with only LoRA fine-tuning. A semantic keyframe selection strategy, together with aesthetic preference filtering, further ensures informative and stable memory throughout generation. Moreover, the proposed framework naturally accommodates smooth shot transitions and customized story generation applications. To facilitate evaluation, we introduce ST-Bench, a diverse benchmark for multi-shot video storytelling. Extensive experiments demonstrate that StoryMem achieves superior cross-shot consistency over previous methods while preserving high aesthetic quality and prompt adherence, marking a significant step toward coherent minute-long video storytelling.
StoryMem enhances multi-shot video generation with cinematic quality and long-range consistency using a memory bank and pre-trained single-shot video diffusion models.
StoryMem Memory-to-Video (M2V) memory bank keyframes latent concatenation negative RoPE shifts LoRA fine-tuning semantic keyframe selection aesthetic preference filtering shot transitions ST-Bench cross-shot consistency aesthetic quality prompt adherence