Imagination Helps Visual Reasoning, But Not Yet in Latent Space
Latent visual reasoning aims to mimic human's imagination process by meditating through hidden states of Multimodal Large Language Models. While recognized as a promising paradigm for visual reasoning, the underlying mechanisms driving its effectiveness remain unclear. Motivated to demystify the true source of its efficacy, we investigate the validity of latent reasoning using Causal Mediation Analysis. We model the process as a causal chain: the input as the treatment, the latent tokens as the mediator, and the final answer as the outcome. Our findings uncover two critical disconnections: (a) Input-Latent Disconnect: dramatic perturbations on the input result in negligible changes to the latent tokens, suggesting that latent tokens do not effectively attend to the input sequence. (b) Latent-Answer Disconnect: perturbations on the latent tokens yield minimal impact on the final answer, indicating the limited causal effect latent tokens imposing on the outcome. Furthermore, extensive probing analysis reveals that latent tokens encode limited visual information and exhibit high similarity. Consequently, we challenge the necessity of latent reasoning and propose a straightforward alternative named CapImagine, which teaches the model to explicitly imagine using text. Experiments on vision-centric benchmarks show that CapImagine significantly outperforms complex latent-space baselines, highlighting the superior potential of visual reasoning through explicit imagination.
69a112ffa13deaa44944896f
You Li
69a112ffa13deaa449448970
Chi Chen
69a112ffa13deaa449448971
Yanghao Li
69a112ffa13deaa449448972
Fanhu Zeng
claimed_verified
2026-03-16T10:45:13.461Z
667b7388580a979b703c3c2b
/avatars/15d42bf75f88f1e2ac112c98cb53bad6.svg
Fanhu Zeng
AuroraZengfh
user
AuroraZengfh
69a112ffa13deaa449448973
Kaiyu Huang
69a112ffa13deaa449448974
Jinan Xu
69a112ffa13deaa449448975
Maosong Sun
Imagination Helps Visual Reasoning, But Not Yet in Latent Space
Latent visual reasoning aims to mimic human's imagination process by meditating through hidden states of Multimodal Large Language Models. While recognized as a promising paradigm for visual reasoning, the underlying mechanisms driving its effectiveness remain unclear. Motivated to demystify the true source of its efficacy, we investigate the validity of latent reasoning using Causal Mediation Analysis. We model the process as a causal chain: the input as the treatment, the latent tokens as the mediator, and the final answer as the outcome. Our findings uncover two critical disconnections: (a) Input-Latent Disconnect: dramatic perturbations on the input result in negligible changes to the latent tokens, suggesting that latent tokens do not effectively attend to the input sequence. (b) Latent-Answer Disconnect: perturbations on the latent tokens yield minimal impact on the final answer, indicating the limited causal effect latent tokens imposing on the outcome. Furthermore, extensive probing analysis reveals that latent tokens encode limited visual information and exhibit high similarity. Consequently, we challenge the necessity of latent reasoning and propose a straightforward alternative named CapImagine, which teaches the model to explicitly imagine using text. Experiments on vision-centric benchmarks show that CapImagine significantly outperforms complex latent-space baselines, highlighting the superior potential of visual reasoning through explicit imagination.
Research reveals that latent visual reasoning in multimodal models suffers from input-latent and latent-answer disconnects, leading to the proposal of CapImagine, a text-based approach that outperforms complex latent-space methods.
Multimodal Large Language Models causal mediation analysis latent tokens visual reasoning input-latent disconnect latent-answer disconnect CapImagine