MME-CC: A Challenging Multi-Modal Evaluation Benchmark of Cognitive
  Capacity
As reasoning models scale rapidly, the essential role of multimodality in
human cognition has come into sharp relief, driving a growing need to probe
vision-centric cognitive behaviors. Yet, existing multimodal benchmarks either
overemphasize textual reasoning or fall short of systematically capturing
vision-centric cognitive behaviors, leaving the cognitive capacity of MLLMs
insufficiently assessed. To address this limitation, we introduce MME-CC
(Multi-Modal Evaluation benchmark of Cognitive Capacity), a vision-grounded
benchmark that organizes 11 representative reasoning tasks into three
fundamental categories of visual information: spatial, geometric, and
knowledge-based reasoning, and provides fine-grained analyses of MLLMs'
cognitive capacity across these dimensions. Based on MME-CC, we conduct
extensive experiments over 16 representative MLLMs. Our study reveals that
closed-source models currently lead overall (e.g., 42.66 for Gemini-2.5-Pro vs.
30.45 for GLM-4.5V), while spatial and geometric reasoning remain broadly weak
(less than or equal to 30%). We further identify common error patterns,
including orientation mistakes, fragile cross-view identity persistence, and
poor adherence to counterfactual instructions, and observe that
Chain-of-Thought typically follows a three-stage process (extract -> reason ->
verify) with heavy reliance on visual extraction. We hope this work catalyzes a
shift toward treating the cognitive capacity of MLLMs as central to both
evaluation and model design.
690c0d0560494e4fa7675623
Kaiyuan Zhang
690c0d0560494e4fa7675624
Chenghao Yang
690c0d0560494e4fa7675625
Zhoufutu Wen
690c0d0560494e4fa7675626
Sihang Yuan
690c0d0560494e4fa7675627
Qiuyue Wang
690c0d0560494e4fa7675628
Chaoyi Huang
690c0d0560494e4fa7675629
Guosheng Zhu
690c0d0560494e4fa767562a
He Wang
690c0d0560494e4fa767562b
Huawenyu Lu
690c0d0560494e4fa767562c
Jianing Wen
690c0d0560494e4fa767562d
Jianpeng Jiao
690c0d0560494e4fa767562e
Lishu Luo
690c0d0560494e4fa767562f
Longxiang Liu
690c0d0560494e4fa7675630
Sijin Wu
690c0d0560494e4fa7675631
Xiaolei Zhu
690c0d0560494e4fa7675632
Xuanliang Zhang
690c0d0560494e4fa7675633
Ge Zhang
claimed_verified
2025-11-18T14:09:52.538Z
638efcf4c67af472d316d424
/avatars/97a57859d7d87a3a8f1bb41d32a72bc2.svg
Ge Zhang
zhangysk
user
zhangysk
690c0d0560494e4fa7675634
Yi Lin
690c0d0560494e4fa7675635
Guang Shi
690c0d0560494e4fa7675636
Chaoyou Fu
690c0d0560494e4fa7675637
Wenhao Huang
MME-CC: A Challenging Multi-Modal Evaluation Benchmark of Cognitive
  Capacity
As reasoning models scale rapidly, the essential role of multimodality in
human cognition has come into sharp relief, driving a growing need to probe
vision-centric cognitive behaviors. Yet, existing multimodal benchmarks either
overemphasize textual reasoning or fall short of systematically capturing
vision-centric cognitive behaviors, leaving the cognitive capacity of MLLMs
insufficiently assessed. To address this limitation, we introduce MME-CC
(Multi-Modal Evaluation benchmark of Cognitive Capacity), a vision-grounded
benchmark that organizes 11 representative reasoning tasks into three
fundamental categories of visual information: spatial, geometric, and
knowledge-based reasoning, and provides fine-grained analyses of MLLMs'
cognitive capacity across these dimensions. Based on MME-CC, we conduct
extensive experiments over 16 representative MLLMs. Our study reveals that
closed-source models currently lead overall (e.g., 42.66 for Gemini-2.5-Pro vs.
30.45 for GLM-4.5V), while spatial and geometric reasoning remain broadly weak
(less than or equal to 30%). We further identify common error patterns,
including orientation mistakes, fragile cross-view identity persistence, and
poor adherence to counterfactual instructions, and observe that
Chain-of-Thought typically follows a three-stage process (extract -> reason ->
verify) with heavy reliance on visual extraction. We hope this work catalyzes a
shift toward treating the cognitive capacity of MLLMs as central to both
evaluation and model design.
MME-CC is a vision-grounded benchmark that evaluates multimodal large language models' cognitive capacity across spatial, geometric, and knowledge-based reasoning tasks, revealing weaknesses in spatial and geometric reasoning.
multimodal benchmarks vision-centric cognitive behaviors MME-CC visual information spatial reasoning geometric reasoning knowledge-based reasoning MLLMs cognitive capacity closed-source models orientation mistakes cross-view identity persistence counterfactual instructions Chain-of-Thought