Training Data Efficiency in Multimodal Process Reward Models
Multimodal Process Reward Models (MPRMs) are central to step-level supervision for visual reasoning in MLLMs. Training MPRMs typically requires large-scale Monte Carlo (MC)-annotated corpora, incurring substantial training cost. This paper studies the data efficiency for MPRM training.Our preliminary experiments reveal that MPRM training quickly saturates under random subsampling of the training data, indicating substantial redundancy within existing MC-annotated corpora.To explain this, we formalize a theoretical framework and reveal that informative gradient updates depend on two factors: label mixtures of positive/negative steps and label reliability (average MC scores of positive steps). Guided by these insights, we propose the Balanced-Information Score (BIS), which prioritizes both mixture and reliability based on existing MC signals at the rollout level, without incurring any additional cost. Across two backbones (InternVL2.5-8B and Qwen2.5-VL-7B) on VisualProcessBench, BIS-selected subsets consistently match and even surpass the full-data performance at small fractions. Notably, the BIS subset reaches full-data performance using only 10% of the training data, improving over random subsampling by a relative 4.1%.
698412bbe34659da7e1f4e04
Jinyuan Li
claimed_verified
2026-02-05T10:53:57.944Z
64fc20d899123d7698a30e61
/avatars/9231982cf70a0689f50accedf1004702.svg
Jinyuan Li
jinyuan222
user
jinyuan222
698412bbe34659da7e1f4e05
Chengsong Huang
claimed_verified
2026-02-06T18:55:50.153Z
62ea79dd01ed9b0e8f61ccd3
/avatars/70af83e0e267be39fcd5f23b85e2dafa.svg
Chengsong Huang
ChengsongHuang
user
ChengsongHuang
698412bbe34659da7e1f4e06
Langlin Huang
698412bbe34659da7e1f4e07
Shaoyang Xu
698412bbe34659da7e1f4e08
Haolin Liu
698412bbe34659da7e1f4e09
Wenxuan Zhang
698412bbe34659da7e1f4e0a
Jiaxin Huang
Training Data Efficiency in Multimodal Process Reward Models
Multimodal Process Reward Models (MPRMs) are central to step-level supervision for visual reasoning in MLLMs. Training MPRMs typically requires large-scale Monte Carlo (MC)-annotated corpora, incurring substantial training cost. This paper studies the data efficiency for MPRM training.Our preliminary experiments reveal that MPRM training quickly saturates under random subsampling of the training data, indicating substantial redundancy within existing MC-annotated corpora.To explain this, we formalize a theoretical framework and reveal that informative gradient updates depend on two factors: label mixtures of positive/negative steps and label reliability (average MC scores of positive steps). Guided by these insights, we propose the Balanced-Information Score (BIS), which prioritizes both mixture and reliability based on existing MC signals at the rollout level, without incurring any additional cost. Across two backbones (InternVL2.5-8B and Qwen2.5-VL-7B) on VisualProcessBench, BIS-selected subsets consistently match and even surpass the full-data performance at small fractions. Notably, the BIS subset reaches full-data performance using only 10% of the training data, improving over random subsampling by a relative 4.1%.
Training multimodal process reward models efficiently through balanced-information scoring that prioritizes label mixture and reliability while achieving full-data performance with only 10% of training data.
Multimodal Process Reward Models Monte Carlo-annotated corpora VisualProcessBench Balanced-Information Score label mixtures label reliability gradient updates data efficiency