Benchmark Designers Should "Train on the Test Set" to Expose Exploitable
  Non-Visual Shortcuts
Robust benchmarks are crucial for evaluating Multimodal Large Language Models
(MLLMs). Yet we find that models can ace many multimodal benchmarks without
strong visual understanding, instead exploiting biases, linguistic priors, and
superficial patterns. This is especially problematic for vision-centric
benchmarks that are meant to require visual inputs. We adopt a diagnostic
principle for benchmark design: if a benchmark can be gamed, it will be.
Designers should therefore try to ``game'' their own benchmarks first, using
diagnostic and debiasing procedures to systematically identify and mitigate
non-visual biases. Effective diagnosis requires directly ``training on the test
set'' -- probing the released test set for its intrinsic, exploitable patterns.
  We operationalize this standard with two components. First, we diagnose
benchmark susceptibility using a ``Test-set Stress-Test'' (TsT) methodology.
Our primary diagnostic tool involves fine-tuning a powerful Large Language
Model via k-fold cross-validation on exclusively the non-visual, textual
inputs of the test set to reveal shortcut performance and assign each sample a
bias score s(x). We complement this with a lightweight Random Forest-based
diagnostic operating on hand-crafted features for fast, interpretable auditing.
Second, we debias benchmarks by filtering high-bias samples using an
``Iterative Bias Pruning'' (IBP) procedure. Applying this framework to four
benchmarks -- VSI-Bench, CV-Bench, MMMU, and VideoMME -- we uncover pervasive
non-visual biases. As a case study, we apply our full framework to create
VSI-Bench-Debiased, demonstrating reduced non-visual solvability and a wider
vision-blind performance gap than the original.
690d627fad2597bf6c464d77
Ellis Brown
claimed_verified
2025-11-10T09:31:30.730Z
626dc5105f7327906f0b2a4e
https://cdn-avatars.huggingface.co/v1/production/uploads/626dc5105f7327906f0b2a4e/QCSzuwYqsv8ozRnusVb-F.jpeg
Ellis Brown
ellisbrown
user
ellisbrown
690d627fad2597bf6c464d78
Jihan Yang
claimed_verified
2025-11-17T10:32:08.072Z
6304baf041387c7f1177a5d2
https://cdn-avatars.huggingface.co/v1/production/uploads/6304baf041387c7f1177a5d2/cQgCR8AsrMUaF2QVh97I9.jpeg
Jihan Yang
jihanyang
user
jihanyang
690d627fad2597bf6c464d79
Shusheng Yang
690d627fad2597bf6c464d7a
Rob Fergus
690d627fad2597bf6c464d7b
Saining Xie
claimed_verified
2025-11-11T19:51:32.046Z
6596422646624a86ff3b3bda
/avatars/216e12b77e45ac5f1fa20932f5745411.svg
Saining Xie
sainx
user
sainx
Benchmark Designers Should "Train on the Test Set" to Expose Exploitable
  Non-Visual Shortcuts
Robust benchmarks are crucial for evaluating Multimodal Large Language Models
(MLLMs). Yet we find that models can ace many multimodal benchmarks without
strong visual understanding, instead exploiting biases, linguistic priors, and
superficial patterns. This is especially problematic for vision-centric
benchmarks that are meant to require visual inputs. We adopt a diagnostic
principle for benchmark design: if a benchmark can be gamed, it will be.
Designers should therefore try to ``game'' their own benchmarks first, using
diagnostic and debiasing procedures to systematically identify and mitigate
non-visual biases. Effective diagnosis requires directly ``training on the test
set'' -- probing the released test set for its intrinsic, exploitable patterns.
  We operationalize this standard with two components. First, we diagnose
benchmark susceptibility using a ``Test-set Stress-Test'' (TsT) methodology.
Our primary diagnostic tool involves fine-tuning a powerful Large Language
Model via k-fold cross-validation on exclusively the non-visual, textual
inputs of the test set to reveal shortcut performance and assign each sample a
bias score s(x). We complement this with a lightweight Random Forest-based
diagnostic operating on hand-crafted features for fast, interpretable auditing.
Second, we debias benchmarks by filtering high-bias samples using an
``Iterative Bias Pruning'' (IBP) procedure. Applying this framework to four
benchmarks -- VSI-Bench, CV-Bench, MMMU, and VideoMME -- we uncover pervasive
non-visual biases. As a case study, we apply our full framework to create
VSI-Bench-Debiased, demonstrating reduced non-visual solvability and a wider
vision-blind performance gap than the original.
A framework for diagnosing and debiasing multimodal benchmarks reveals and mitigates non-visual biases, improving the robustness of Multimodal Large Language Models.
multimodal large language models MLLMs vision-centric benchmarks diagnostic principle test-set stress-test TsT fine-tuning k-fold cross-validation bias score random forest iterative bias pruning IBP VSI-Bench CV-Bench MMMU VideoMME VSI-Bench-Debiased