AdaptMMBench: Benchmarking Adaptive Multimodal Reasoning for Mode Selection and Reasoning Process
Adaptive multimodal reasoning has emerged as a promising frontier in Vision-Language Models (VLMs), aiming to dynamically modulate between tool-augmented visual reasoning and text reasoning to enhance both effectiveness and efficiency. However, existing evaluations rely on static difficulty labels and simplistic metrics, which fail to capture the dynamic nature of difficulty relative to varying model capacities. Consequently, they obscure the distinction between adaptive mode selection and general performance while neglecting fine-grained process analyses. In this paper, we propose AdaptMMBench, a comprehensive benchmark for adaptive multimodal reasoning across five domains: real-world, OCR, GUI, knowledge, and math, encompassing both direct perception and complex reasoning tasks. AdaptMMBench utilizes a Matthews Correlation Coefficient (MCC) metric to evaluate the selection rationality of different reasoning modes, isolating this meta-cognition ability by dynamically identifying task difficulties based on models' capability boundaries. Moreover, AdaptMMBench facilitates multi-dimensional process evaluation across key step coverage, tool effectiveness, and computational efficiency. Our evaluation reveals that while adaptive mode selection scales with model capacity, it notably decouples from final accuracy. Conversely, key step coverage aligns with performance, though tool effectiveness remains highly inconsistent across model architectures.
6982b0439084cb4f0ecb562e
Xintong Zhang
claimed_verified
2026-02-04T12:29:02.801Z
66b5b0ceef3f3fc987957343
/avatars/915de4d13789cd50f0cb61a9bf5fce80.svg
Xintong Zhang
xintongzhang
user
xintongzhang
6982b0439084cb4f0ecb562f
Xiaowen Zhang
6982b0439084cb4f0ecb5630
Jongrong Wu
6982b0439084cb4f0ecb5631
Zhi Gao
6982b0439084cb4f0ecb5632
Shilin Yan
6982b0439084cb4f0ecb5633
Zhenxin Diao
6982b0439084cb4f0ecb5634
Kunpeng Gao
6982b0439084cb4f0ecb5635
Xuanyan Chen
6982b0439084cb4f0ecb5636
Yuwei Wu
6982b0439084cb4f0ecb5637
Yunde Jia
6982b0439084cb4f0ecb5638
Qing Li
AdaptMMBench: Benchmarking Adaptive Multimodal Reasoning for Mode Selection and Reasoning Process
Adaptive multimodal reasoning has emerged as a promising frontier in Vision-Language Models (VLMs), aiming to dynamically modulate between tool-augmented visual reasoning and text reasoning to enhance both effectiveness and efficiency. However, existing evaluations rely on static difficulty labels and simplistic metrics, which fail to capture the dynamic nature of difficulty relative to varying model capacities. Consequently, they obscure the distinction between adaptive mode selection and general performance while neglecting fine-grained process analyses. In this paper, we propose AdaptMMBench, a comprehensive benchmark for adaptive multimodal reasoning across five domains: real-world, OCR, GUI, knowledge, and math, encompassing both direct perception and complex reasoning tasks. AdaptMMBench utilizes a Matthews Correlation Coefficient (MCC) metric to evaluate the selection rationality of different reasoning modes, isolating this meta-cognition ability by dynamically identifying task difficulties based on models' capability boundaries. Moreover, AdaptMMBench facilitates multi-dimensional process evaluation across key step coverage, tool effectiveness, and computational efficiency. Our evaluation reveals that while adaptive mode selection scales with model capacity, it notably decouples from final accuracy. Conversely, key step coverage aligns with performance, though tool effectiveness remains highly inconsistent across model architectures.
AdaptMMBench presents a comprehensive benchmark for evaluating adaptive multimodal reasoning in Vision-Language Models, measuring reasoning mode selection rationality through dynamic difficulty assessment and multi-dimensional process evaluation.
Vision-Language Models adaptive multimodal reasoning Matthews Correlation Coefficient task difficulty model capacity reasoning mode selection multi-dimensional process evaluation key step coverage tool effectiveness computational efficiency