CMI-RewardBench: Evaluating Music Reward Models with Compositional Multimodal Instruction
While music generation models have evolved to handle complex multimodal inputs mixing text, lyrics, and reference audio, evaluation mechanisms have lagged behind. In this paper, we bridge this critical gap by establishing a comprehensive ecosystem for music reward modeling under Compositional Multimodal Instruction (CMI), where the generated music may be conditioned on text descriptions, lyrics, and audio prompts. We first introduce CMI-Pref-Pseudo, a large-scale preference dataset comprising 110k pseudo-labeled samples, and CMI-Pref, a high-quality, human-annotated corpus tailored for fine-grained alignment tasks. To unify the evaluation landscape, we propose CMI-RewardBench, a unified benchmark that evaluates music reward models on heterogeneous samples across musicality, text-music alignment, and compositional instruction alignment. Leveraging these resources, we develop CMI reward models (CMI-RMs), a parameter-efficient reward model family capable of processing heterogeneous inputs. We evaluate their correlation with human judgments scores on musicality and alignment on CMI-Pref along with previous datasets. Further experiments demonstrate that CMI-RM not only correlates strongly with human judgments, but also enables effective inference-time scaling via top-k filtering. The necessary training data, benchmarks, and reward models are publicly available.
69a774bc2833ff1122ff63c1
Yinghao Ma
69a774bc2833ff1122ff63c2
Haiwen Xia
69a774bc2833ff1122ff63c3
Hewei Gao
69a774bc2833ff1122ff63c4
Weixiong Chen
claimed_verified
2026-03-05T09:59:17.782Z
6628452bb11cb848a44a49c8
/avatars/e9295bc28549c8501588430370f88ea5.svg
Weixiong Chen
Leon299
user
Leon299
69a774bc2833ff1122ff63c5
Yuxin Ye
69a774bc2833ff1122ff63c6
Yuchen Yang
69a774bc2833ff1122ff63c7
Sungkyun Chang
69a774bc2833ff1122ff63c8
Mingshuo Ding
69a774bc2833ff1122ff63c9
Yizhi Li
claimed_verified
2026-04-07T08:48:44.466Z
6382252f54421460665ec501
https://cdn-avatars.huggingface.co/v1/production/uploads/6382252f54421460665ec501/gW9fev3T5QPcNq4f9hqB1.jpeg
Yizhi Li
yizhilll
user
yizhilll
69a774bc2833ff1122ff63ca
Ruibin Yuan
69a774bc2833ff1122ff63cb
Simon Dixon
69a774bc2833ff1122ff63cc
Emmanouil Benetos
CMI-RewardBench: Evaluating Music Reward Models with Compositional Multimodal Instruction
While music generation models have evolved to handle complex multimodal inputs mixing text, lyrics, and reference audio, evaluation mechanisms have lagged behind. In this paper, we bridge this critical gap by establishing a comprehensive ecosystem for music reward modeling under Compositional Multimodal Instruction (CMI), where the generated music may be conditioned on text descriptions, lyrics, and audio prompts. We first introduce CMI-Pref-Pseudo, a large-scale preference dataset comprising 110k pseudo-labeled samples, and CMI-Pref, a high-quality, human-annotated corpus tailored for fine-grained alignment tasks. To unify the evaluation landscape, we propose CMI-RewardBench, a unified benchmark that evaluates music reward models on heterogeneous samples across musicality, text-music alignment, and compositional instruction alignment. Leveraging these resources, we develop CMI reward models (CMI-RMs), a parameter-efficient reward model family capable of processing heterogeneous inputs. We evaluate their correlation with human judgments scores on musicality and alignment on CMI-Pref along with previous datasets. Further experiments demonstrate that CMI-RM not only correlates strongly with human judgments, but also enables effective inference-time scaling via top-k filtering. The necessary training data, benchmarks, and reward models are publicly available.
A comprehensive ecosystem for music reward modeling under compositional multimodal instruction is established through large-scale datasets, unified benchmark, and parameter-efficient reward models that correlate well with human judgments.
music reward modeling Compositional Multimodal Instruction preference dataset pseudo-labeled samples human-annotated corpus unified benchmark reward models parameter-efficient top-k filtering