Music Flamingo: Scaling Music Understanding in Audio Language Models
We introduce Music Flamingo, a novel large audio-language model designed to advance music (including song) understanding in foundational audio models. While audio-language research has progressed rapidly, music remains challenging due to its dynamic, layered, and information-dense nature. Progress has been further limited by the difficulty of scaling open audio understanding models, primarily because of the scarcity of high-quality music data and annotations. As a result, prior models are restricted to producing short, high-level captions, answering only surface-level questions, and showing limited generalization across diverse musical cultures. To address these challenges, we curate MF-Skills, a large-scale dataset labeled through a multi-stage pipeline that yields rich captions and question-answer pairs covering harmony, structure, timbre, lyrics, and cultural context. We fine-tune an enhanced Audio Flamingo 3 backbone on MF-Skills and further strengthen multiple skills relevant to music understanding. To improve the model's reasoning abilities, we introduce a post-training recipe: we first cold-start with MF-Think, a novel chain-of-thought dataset grounded in music theory, followed by GRPO-based reinforcement learning with custom rewards. Music Flamingo achieves state-of-the-art results across 10+ benchmarks for music understanding and reasoning, establishing itself as a generalist and musically intelligent audio-language model. Beyond strong empirical results, Music Flamingo sets a new standard for advanced music understanding by demonstrating how models can move from surface-level recognition toward layered, human-like perception of songs. We believe this work provides both a benchmark and a foundation for the community to build the next generation of models that engage with music as meaningfully as humans do.
6916aa949a1d7af6ca2f019e
Sreyan Ghosh
claimed_verified
2025-11-14T14:22:19.454Z
62c9664eb34e600d7eaa4beb
/avatars/ca23ecdec2d31c99ecce97d9b180ae0c.svg
Ghosh
Sreyan88
user
Sreyan88
6916aa949a1d7af6ca2f019f
Arushi Goel
6916aa949a1d7af6ca2f01a0
Lasha Koroshinadze
claimed_verified
2026-01-28T11:33:20.519Z
63fc96a0b9db84750cec216f
/avatars/eb8cb2518260de4851f8f6aa8bc5508c.svg
Lasha Koroshinadze
lashahub
user
lashahub
6916aa949a1d7af6ca2f01a1
Sang-gil Lee
claimed_verified
2025-12-31T20:58:36.678Z
628dba7ea13c4b25ff786ec6
https://cdn-avatars.huggingface.co/v1/production/uploads/1671164940704-628dba7ea13c4b25ff786ec6.jpeg
Sang-gil Lee
L0SG
user
L0SG
6916aa949a1d7af6ca2f01a2
Zhifeng Kong
6916aa949a1d7af6ca2f01a3
Joao Felipe Santos
6916aa949a1d7af6ca2f01a4
Ramani Duraiswami
6916aa949a1d7af6ca2f01a5
Dinesh Manocha
6916aa949a1d7af6ca2f01a6
Wei Ping
6916aa949a1d7af6ca2f01a7
Mohammad Shoeybi
6916aa949a1d7af6ca2f01a8
Bryan Catanzaro
Music Flamingo: Scaling Music Understanding in Audio Language Models
We introduce Music Flamingo, a novel large audio-language model designed to advance music (including song) understanding in foundational audio models. While audio-language research has progressed rapidly, music remains challenging due to its dynamic, layered, and information-dense nature. Progress has been further limited by the difficulty of scaling open audio understanding models, primarily because of the scarcity of high-quality music data and annotations. As a result, prior models are restricted to producing short, high-level captions, answering only surface-level questions, and showing limited generalization across diverse musical cultures. To address these challenges, we curate MF-Skills, a large-scale dataset labeled through a multi-stage pipeline that yields rich captions and question-answer pairs covering harmony, structure, timbre, lyrics, and cultural context. We fine-tune an enhanced Audio Flamingo 3 backbone on MF-Skills and further strengthen multiple skills relevant to music understanding. To improve the model's reasoning abilities, we introduce a post-training recipe: we first cold-start with MF-Think, a novel chain-of-thought dataset grounded in music theory, followed by GRPO-based reinforcement learning with custom rewards. Music Flamingo achieves state-of-the-art results across 10+ benchmarks for music understanding and reasoning, establishing itself as a generalist and musically intelligent audio-language model. Beyond strong empirical results, Music Flamingo sets a new standard for advanced music understanding by demonstrating how models can move from surface-level recognition toward layered, human-like perception of songs. We believe this work provides both a benchmark and a foundation for the community to build the next generation of models that engage with music as meaningfully as humans do.
Music Flamingo, a large audio-language model, advances music understanding through fine-tuning on a rich dataset and post-training with novel methods, achieving state-of-the-art results across various benchmarks.
audio-language model Music Flamingo MF-Skills multi-stage pipeline Audio Flamingo 3 fine-tuning MF-Think chain-of-thought dataset GRPO-based reinforcement learning custom rewards music understanding reasoning benchmarks human-like perception