Reasoning Within the Mind: Dynamic Multimodal Interleaving in Latent Space
Recent advancements in Multimodal Large Language Models (MLLMs) have significantly enhanced cross-modal understanding and reasoning by incorporating Chain-of-Thought (CoT) reasoning in the semantic space. Building upon this, recent studies extend the CoT mechanism to the visual modality, enabling models to integrate visual information during reasoning through external tools or explicit image generation. However, these methods remain dependent on explicit step-by-step reasoning, unstable perception-reasoning interaction and notable computational overhead. Inspired by human cognition, we posit that thinking unfolds not linearly but through the dynamic interleaving of reasoning and perception within the mind. Motivated by this perspective, we propose DMLR, a test-time Dynamic Multimodal Latent Reasoning framework that employs confidence-guided latent policy gradient optimization to refine latent think tokens for in-depth reasoning. Furthermore, a Dynamic Visual Injection Strategy is introduced, which retrieves the most relevant visual features at each latent think token and updates the set of best visual patches. The updated patches are then injected into latent think token to achieve dynamic visual-textual interleaving. Experiments across seven multimodal reasoning benchmarks and various model architectures demonstrate that DMLR significantly improves reasoning and perception performance while maintaining high inference efficiency.
69422ade5d5b2dc10527489b
Chengzhi Liu
claimed_verified
2025-12-22T11:01:02.483Z
65e71f6bcd3df9b0f6b2678b
https://cdn-avatars.huggingface.co/v1/production/uploads/65e71f6bcd3df9b0f6b2678b/8mdex6eI80TGezot_GGI3.jpeg
Chengzhi Liu
LCZZZZ
user
LCZZZZ
69422ade5d5b2dc10527489c
Yuzhe Yang
69422ade5d5b2dc10527489d
Yue Fan
69422ade5d5b2dc10527489e
Qingyue Wei
69422ade5d5b2dc10527489f
Sheng Liu
69422ade5d5b2dc1052748a0
Xin Eric Wang
Reasoning Within the Mind: Dynamic Multimodal Interleaving in Latent Space
Recent advancements in Multimodal Large Language Models (MLLMs) have significantly enhanced cross-modal understanding and reasoning by incorporating Chain-of-Thought (CoT) reasoning in the semantic space. Building upon this, recent studies extend the CoT mechanism to the visual modality, enabling models to integrate visual information during reasoning through external tools or explicit image generation. However, these methods remain dependent on explicit step-by-step reasoning, unstable perception-reasoning interaction and notable computational overhead. Inspired by human cognition, we posit that thinking unfolds not linearly but through the dynamic interleaving of reasoning and perception within the mind. Motivated by this perspective, we propose DMLR, a test-time Dynamic Multimodal Latent Reasoning framework that employs confidence-guided latent policy gradient optimization to refine latent think tokens for in-depth reasoning. Furthermore, a Dynamic Visual Injection Strategy is introduced, which retrieves the most relevant visual features at each latent think token and updates the set of best visual patches. The updated patches are then injected into latent think token to achieve dynamic visual-textual interleaving. Experiments across seven multimodal reasoning benchmarks and various model architectures demonstrate that DMLR significantly improves reasoning and perception performance while maintaining high inference efficiency.
A dynamic multimodal latent reasoning framework improves cross-modal reasoning and perception performance by interleaving reasoning and perception using confidence-guided optimization and dynamic visual injection.
Chain-of-Thought CoT MLLMs multimodal reasoning semantic space visual modality external tools image generation latent policy gradient optimization latent think tokens Dynamic Visual Injection Strategy multimodal benchmarks