Cognitive Mismatch in Multimodal Large Language Models for Discrete Symbol Understanding
While Multimodal Large Language Models (MLLMs) have achieved remarkable success in interpreting natural scenes, their ability to process discrete symbols -- the fundamental building blocks of human cognition -- remains a critical open question. Unlike continuous visual data, symbols such as mathematical formulas, chemical structures, and linguistic characters require precise, deeper interpretation. This paper introduces a comprehensive benchmark to evaluate how top-tier MLLMs navigate these "discrete semantic spaces" across five domains: language, culture, mathematics, physics, and chemistry. Our investigation uncovers a counterintuitive phenomenon: models often fail at basic symbol recognition yet succeed in complex reasoning tasks, suggesting they rely on linguistic probability rather than true visual perception. By exposing this "cognitive mismatch", we highlight a significant gap in current AI capabilities: the struggle to truly perceive and understand the symbolic languages that underpin scientific discovery and abstract thought. This work offers a roadmap for developing more rigorous, human-aligned intelligent systems.
69bcd980081b9ad49af572f3
Yinghui Li
claimed_verified
2026-03-23T10:52:21.786Z
692e96fd7bf6e835a968d7c6
/avatars/c18a79cb53df74434f2d04ba145fe980.svg
Yinghui Li
liyinghui1998
user
liyinghui1998
69bcd980081b9ad49af572f4
Jiayi Kuang
69bcd980081b9ad49af572f5
Peng Xing
69bcd980081b9ad49af572f6
Daixian Liu
69bcd980081b9ad49af572f7
Junnan Dong
69bcd980081b9ad49af572f8
Shu-Yu Guo
69bcd980081b9ad49af572f9
Yangning Li
69bcd980081b9ad49af572fa
Qingyu Zhou
69bcd980081b9ad49af572fb
Wenhao Jiang
claimed_verified
2026-03-23T10:52:25.648Z
6359ed436a619540808a559a
/avatars/5e9f0896ad7ea31a9f08b7ea8de862a7.svg
wh
cswhjiang
user
cswhjiang
69bcd980081b9ad49af572fc
Hai-Tao Zheng
69bcd980081b9ad49af572fd
Ying Shen
69bcd980081b9ad49af572fe
Liang Lin
69bcd980081b9ad49af572ff
Philip S. Yu
Cognitive Mismatch in Multimodal Large Language Models for Discrete Symbol Understanding
While Multimodal Large Language Models (MLLMs) have achieved remarkable success in interpreting natural scenes, their ability to process discrete symbols -- the fundamental building blocks of human cognition -- remains a critical open question. Unlike continuous visual data, symbols such as mathematical formulas, chemical structures, and linguistic characters require precise, deeper interpretation. This paper introduces a comprehensive benchmark to evaluate how top-tier MLLMs navigate these "discrete semantic spaces" across five domains: language, culture, mathematics, physics, and chemistry. Our investigation uncovers a counterintuitive phenomenon: models often fail at basic symbol recognition yet succeed in complex reasoning tasks, suggesting they rely on linguistic probability rather than true visual perception. By exposing this "cognitive mismatch", we highlight a significant gap in current AI capabilities: the struggle to truly perceive and understand the symbolic languages that underpin scientific discovery and abstract thought. This work offers a roadmap for developing more rigorous, human-aligned intelligent systems.
Top-tier MLLMs demonstrate limited capability in processing discrete symbols despite strong performance in complex reasoning, revealing a cognitive mismatch between visual perception and symbolic understanding.
Multimodal Large Language Models discrete semantic spaces symbol recognition visual perception cognitive mismatch