What Matters in Data Curation for Multimodal Reasoning? Insights from the DCVLR Challenge
We study data curation for multimodal reasoning through the NeurIPS 2025 Data Curation for Vision-Language Reasoning (DCVLR) challenge, which isolates dataset selection by fixing the model and training protocol. Using a compact curated dataset derived primarily from Walton Multimodal Cold Start, our submission placed first in the challenge. Through post-competition ablations, we show that difficulty-based example selection on an aligned base dataset is the dominant driver of performance gains. Increasing dataset size does not reliably improve mean accuracy under the fixed training recipe, but mainly reduces run-to-run variance, while commonly used diversity and synthetic augmentation heuristics provide no additional benefit and often degrade performance. These results characterize DCVLR as a saturation-regime evaluation and highlight the central role of alignment and difficulty in data-efficient multimodal reasoning.
696e6ff15750539a88acc701
Yosub Shin
696e6ff15750539a88acc702
Michael Buriek
696e6ff15750539a88acc703
Boris Sobolev
696e6ff15750539a88acc704
Pavel Bushuyeu
696e6ff15750539a88acc705
Vikas Kumar
696e6ff15750539a88acc706
Haoyang Xu
696e6ff15750539a88acc707
Samuel Watson
696e6ff15750539a88acc708
Igor Molybog
What Matters in Data Curation for Multimodal Reasoning? Insights from the DCVLR Challenge
We study data curation for multimodal reasoning through the NeurIPS 2025 Data Curation for Vision-Language Reasoning (DCVLR) challenge, which isolates dataset selection by fixing the model and training protocol. Using a compact curated dataset derived primarily from Walton Multimodal Cold Start, our submission placed first in the challenge. Through post-competition ablations, we show that difficulty-based example selection on an aligned base dataset is the dominant driver of performance gains. Increasing dataset size does not reliably improve mean accuracy under the fixed training recipe, but mainly reduces run-to-run variance, while commonly used diversity and synthetic augmentation heuristics provide no additional benefit and often degrade performance. These results characterize DCVLR as a saturation-regime evaluation and highlight the central role of alignment and difficulty in data-efficient multimodal reasoning.
Data curation for multimodal reasoning shows that difficulty-based example selection on aligned datasets drives performance gains, while increasing dataset size mainly reduces variance and synthetic augmentation heuristics often degrade performance.
data curation multimodal reasoning dataset selection model training example selection aligned dataset saturation-regime evaluation alignment difficulty