MPJudge: Towards Perceptual Assessment of Music-Induced Paintings
Music induced painting is a unique artistic practice, where visual artworks
are created under the influence of music. Evaluating whether a painting
faithfully reflects the music that inspired it poses a challenging perceptual
assessment task. Existing methods primarily rely on emotion recognition models
to assess the similarity between music and painting, but such models introduce
considerable noise and overlook broader perceptual cues beyond emotion. To
address these limitations, we propose a novel framework for music induced
painting assessment that directly models perceptual coherence between music and
visual art. We introduce MPD, the first large scale dataset of music painting
pairs annotated by domain experts based on perceptual coherence. To better
handle ambiguous cases, we further collect pairwise preference annotations.
Building on this dataset, we present MPJudge, a model that integrates music
features into a visual encoder via a modulation based fusion mechanism. To
effectively learn from ambiguous cases, we adopt Direct Preference Optimization
for training. Extensive experiments demonstrate that our method outperforms
existing approaches. Qualitative results further show that our model more
accurately identifies music relevant regions in paintings.
6912d8eaa644ba07c499c808
Shiqi Jiang
claimed_verified
2025-11-11T19:43:23.985Z
670765da8d82328304a11503
/avatars/2cce012febe32dc195eabb168b58efeb.svg
Shiqi Jiang
Eriiiiiin
user
Eriiiiiin
6912d8eaa644ba07c499c809
Tianyi Liang
claimed_verified
2025-11-11T19:43:20.468Z
62c14609ac1b639c2d87192c
https://cdn-avatars.huggingface.co/v1/production/uploads/1656833489364-noauth.png
SII-liangtianyi
tianyilt
user
tianyilt
6912d8eaa644ba07c499c80a
Changbo Wang
6912d8eaa644ba07c499c80b
Chenhui Li
MPJudge: Towards Perceptual Assessment of Music-Induced Paintings
Music induced painting is a unique artistic practice, where visual artworks
are created under the influence of music. Evaluating whether a painting
faithfully reflects the music that inspired it poses a challenging perceptual
assessment task. Existing methods primarily rely on emotion recognition models
to assess the similarity between music and painting, but such models introduce
considerable noise and overlook broader perceptual cues beyond emotion. To
address these limitations, we propose a novel framework for music induced
painting assessment that directly models perceptual coherence between music and
visual art. We introduce MPD, the first large scale dataset of music painting
pairs annotated by domain experts based on perceptual coherence. To better
handle ambiguous cases, we further collect pairwise preference annotations.
Building on this dataset, we present MPJudge, a model that integrates music
features into a visual encoder via a modulation based fusion mechanism. To
effectively learn from ambiguous cases, we adopt Direct Preference Optimization
for training. Extensive experiments demonstrate that our method outperforms
existing approaches. Qualitative results further show that our model more
accurately identifies music relevant regions in paintings.
A novel framework MPJudge assesses music-induced paintings by integrating music features into a visual encoder using a modulation-based fusion mechanism, outperforming existing emotion recognition models.
perceptual coherence MPD MPJudge modulation-based fusion mechanism Direct Preference Optimization