AutoMV: An Automatic Multi-Agent System for Music Video Generation
Music-to-Video (M2V) generation for full-length songs faces significant challenges. Existing methods produce short, disjointed clips, failing to align visuals with musical structure, beats, or lyrics, and lack temporal consistency. We propose AutoMV, a multi-agent system that generates full music videos (MVs) directly from a song. AutoMV first applies music processing tools to extract musical attributes, such as structure, vocal tracks, and time-aligned lyrics, and constructs these features as contextual inputs for following agents. The screenwriter Agent and director Agent then use this information to design short script, define character profiles in a shared external bank, and specify camera instructions. Subsequently, these agents call the image generator for keyframes and different video generators for "story" or "singer" scenes. A Verifier Agent evaluates their output, enabling multi-agent collaboration to produce a coherent longform MV. To evaluate M2V generation, we further propose a benchmark with four high-level categories (Music Content, Technical, Post-production, Art) and twelve ine-grained criteria. This benchmark was applied to compare commercial products, AutoMV, and human-directed MVs with expert human raters: AutoMV outperforms current baselines significantly across all four categories, narrowing the gap to professional MVs. Finally, we investigate using large multimodal models as automatic MV judges; while promising, they still lag behind human expert, highlighting room for future work.
6940da6f65f1e24a117805d3
Xiaoxuan Tang
6940da6f65f1e24a117805d4
Xinping Lei
6940da6f65f1e24a117805d5
Chaoran Zhu
6940da6f65f1e24a117805d6
Shiyun Chen
6940da6f65f1e24a117805d7
Ruibin Yuan
6940da6f65f1e24a117805d8
Yizhi Li
claimed_verified
2026-04-07T08:49:47.481Z
6382252f54421460665ec501
https://cdn-avatars.huggingface.co/v1/production/uploads/6382252f54421460665ec501/gW9fev3T5QPcNq4f9hqB1.jpeg
Yizhi Li
yizhilll
user
yizhilll
6940da6f65f1e24a117805d9
Changjae Oh
6940da6f65f1e24a117805da
Ge Zhang
claimed_verified
2026-01-12T10:37:20.441Z
638efcf4c67af472d316d424
/avatars/97a57859d7d87a3a8f1bb41d32a72bc2.svg
Ge Zhang
zhangysk
user
zhangysk
6940da6f65f1e24a117805db
Wenhao Huang
6940da6f65f1e24a117805dc
Emmanouil Benetos
6940da6f65f1e24a117805dd
Yang Liu
6940da6f65f1e24a117805de
Jiaheng Liu
6940da6f65f1e24a117805df
Yinghao Ma
AutoMV: An Automatic Multi-Agent System for Music Video Generation
Music-to-Video (M2V) generation for full-length songs faces significant challenges. Existing methods produce short, disjointed clips, failing to align visuals with musical structure, beats, or lyrics, and lack temporal consistency. We propose AutoMV, a multi-agent system that generates full music videos (MVs) directly from a song. AutoMV first applies music processing tools to extract musical attributes, such as structure, vocal tracks, and time-aligned lyrics, and constructs these features as contextual inputs for following agents. The screenwriter Agent and director Agent then use this information to design short script, define character profiles in a shared external bank, and specify camera instructions. Subsequently, these agents call the image generator for keyframes and different video generators for "story" or "singer" scenes. A Verifier Agent evaluates their output, enabling multi-agent collaboration to produce a coherent longform MV. To evaluate M2V generation, we further propose a benchmark with four high-level categories (Music Content, Technical, Post-production, Art) and twelve ine-grained criteria. This benchmark was applied to compare commercial products, AutoMV, and human-directed MVs with expert human raters: AutoMV outperforms current baselines significantly across all four categories, narrowing the gap to professional MVs. Finally, we investigate using large multimodal models as automatic MV judges; while promising, they still lag behind human expert, highlighting room for future work.
AutoMV, a multi-agent system, generates coherent full-length music videos directly from songs, outperforming existing methods and narrowing the gap to professional videos.
multi-agent system music processing musical attributes contextual inputs screenwriter Agent director Agent image generator video generators Verifier Agent benchmark Music Content Technical Post-production Art