Toward Cognitive Supersensing in Multimodal Large Language Model
Multimodal Large Language Models (MLLMs) have achieved remarkable success in open-vocabulary perceptual tasks, yet their ability to solve complex cognitive problems remains limited, especially when visual details are abstract and require visual memory. Current approaches primarily scale Chain-of-Thought (CoT) reasoning in the text space, even when language alone is insufficient for clear and structured reasoning, and largely neglect visual reasoning mechanisms analogous to the human visuospatial sketchpad and visual imagery. To mitigate this deficiency, we introduce Cognitive Supersensing, a novel training paradigm that endows MLLMs with human-like visual imagery capabilities by integrating a Latent Visual Imagery Prediction (LVIP) head that jointly learns sequences of visual cognitive latent embeddings and aligns them with the answer, thereby forming vision-based internal reasoning chains. We further introduce a reinforcement learning stage that optimizes text reasoning paths based on this grounded visual latent. To evaluate the cognitive capabilities of MLLMs, we present CogSense-Bench, a comprehensive visual question answering (VQA) benchmark assessing five cognitive dimensions. Extensive experiments demonstrate that MLLMs trained with Cognitive Supersensing significantly outperform state-of-the-art baselines on CogSense-Bench and exhibit superior generalization on out-of-domain mathematics and science VQA benchmarks, suggesting that internal visual imagery is potentially key to bridging the gap between perceptual recognition and cognitive understanding. We will open-source the CogSense-Bench and our model weights.
6981765bce18b18628096092
Boyi Li
claimed_verified
2026-02-03T10:07:09.962Z
67eb59446b0871640462bfb0
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/fRRrOjm10id376KdVOsAO.png
Boyi Li
Resurgammm
user
Resurgammm
6981765bce18b18628096093
Yifan Shen
claimed_verified
2026-02-09T08:36:23.095Z
65e387095132c2edd193ae49
/avatars/39278e5b026bcbdde88c560fc54018c5.svg
Yifan Shen
SivanSX
user
SivanSX
6981765bce18b18628096094
Yuanzhe Liu
6981765bce18b18628096095
Yifan Xu
6981765bce18b18628096096
Jiateng Liu
6981765bce18b18628096097
Xinzhuo Li
claimed_verified
2026-03-23T11:01:40.500Z
6711d478035ddf7e14b7c537
/avatars/9bf15d8c33c5d544ccb7e3559d029190.svg
Xinzhuo Li
LXZ83419
user
LXZ83419
6981765bce18b18628096098
Zhengyuan Li
6981765bce18b18628096099
Jingyuan Zhu
6981765bce18b1862809609a
Yunhan Zhong
6981765bce18b1862809609b
Fangzhou Lan
6981765bce18b1862809609c
Jianguo Cao
6981765bce18b1862809609d
James M. Rehg
6981765bce18b1862809609e
Heng Ji
6981765bce18b1862809609f
Ismini Lourentzou
claimed_verified
2026-02-12T20:28:56.426Z
678ac3b31cbaa0b4bc295885
/avatars/1244fc1b305c9c6383df9bb5e4707347.svg
Ismini Lourentzou
isminoula
user
isminoula
6981765bce18b186280960a0
Xu Cao
claimed_verified
2026-02-03T10:07:07.281Z
5fc9f05d52770aca770bd3d9
https://cdn-avatars.huggingface.co/v1/production/uploads/5fc9f05d52770aca770bd3d9/axiKCXPYttXEWC-RqCSfh.jpeg
Xu Cao
IrohXu
user
IrohXu
Toward Cognitive Supersensing in Multimodal Large Language Model
Multimodal Large Language Models (MLLMs) have achieved remarkable success in open-vocabulary perceptual tasks, yet their ability to solve complex cognitive problems remains limited, especially when visual details are abstract and require visual memory. Current approaches primarily scale Chain-of-Thought (CoT) reasoning in the text space, even when language alone is insufficient for clear and structured reasoning, and largely neglect visual reasoning mechanisms analogous to the human visuospatial sketchpad and visual imagery. To mitigate this deficiency, we introduce Cognitive Supersensing, a novel training paradigm that endows MLLMs with human-like visual imagery capabilities by integrating a Latent Visual Imagery Prediction (LVIP) head that jointly learns sequences of visual cognitive latent embeddings and aligns them with the answer, thereby forming vision-based internal reasoning chains. We further introduce a reinforcement learning stage that optimizes text reasoning paths based on this grounded visual latent. To evaluate the cognitive capabilities of MLLMs, we present CogSense-Bench, a comprehensive visual question answering (VQA) benchmark assessing five cognitive dimensions. Extensive experiments demonstrate that MLLMs trained with Cognitive Supersensing significantly outperform state-of-the-art baselines on CogSense-Bench and exhibit superior generalization on out-of-domain mathematics and science VQA benchmarks, suggesting that internal visual imagery is potentially key to bridging the gap between perceptual recognition and cognitive understanding. We will open-source the CogSense-Bench and our model weights.
MLLMs equipped with Cognitive Supersensing and Latent Visual Imagery Prediction demonstrate enhanced cognitive reasoning capabilities through integrated visual and textual reasoning pathways.
Multimodal Large Language Models Chain-of-Thought reasoning visual reasoning Latent Visual Imagery Prediction reinforcement learning visual cognitive latent embeddings vision-based internal reasoning chains CogSense-Bench visual question answering