LaViT: Aligning Latent Visual Thoughts for Multi-modal Reasoning
Current multimodal latent reasoning often relies on external supervision (e.g., auxiliary images), ignoring intrinsic visual attention dynamics. In this work, we identify a critical Perception Gap in distillation: student models frequently mimic a teacher's textual output while attending to fundamentally divergent visual regions, effectively relying on language priors rather than grounded perception. To bridge this, we propose LaViT, a framework that aligns latent visual thoughts rather than static embeddings. LaViT compels the student to autoregressively reconstruct the teacher's visual semantics and attention trajectories prior to text generation, employing a curriculum sensory gating mechanism to prevent shortcut learning. Extensive experiments show that LaViT significantly enhances visual grounding, achieving up to +16.9% gains on complex reasoning tasks and enabling a compact 3B model to outperform larger open-source variants and proprietary models like GPT-4o.
6969b6df32f0333869ff94a0
Linquan Wu
claimed_verified
2026-01-19T09:24:01.991Z
6627610bef7dd857cc5872cd
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/PBHmlzqmikkdCY9ftuT5r.png
Wu Linquan
Svard
user
Svard
6969b6df32f0333869ff94a1
Tianxiang Jiang
admin_assigned
2026-01-20T20:55:08.014Z
6744754ff9940208b97a6a9a
https://cdn-avatars.huggingface.co/v1/production/uploads/6744754ff9940208b97a6a9a/PRG6_0jAfsj0uoUJvKyWf.png
Tianxiang Jiang
Eurayka
user
Eurayka
6969b6df32f0333869ff94a2
Yifei Dong
claimed_verified
2026-01-21T09:25:05.146Z
696f9e498df90fe45259ad24
/avatars/8c4ee7a8175a746a76c8b8a8ff9510dc.svg
Yifei Dong
Yifei1314
user
Yifei1314
6969b6df32f0333869ff94a3
Haoyu Yang
admin_assigned
2026-01-16T15:48:11.448Z
679770cc0972df3a95b9d6ba
https://cdn-avatars.huggingface.co/v1/production/uploads/679770cc0972df3a95b9d6ba/bAC6fGb1pXeGUSu4tG3YJ.jpeg
Haoyu Yang
afunnyhy
user
afunnyhy
6969b6df32f0333869ff94a4
Fengji Zhang
6969b6df32f0333869ff94a5
Shichaang Meng
6969b6df32f0333869ff94a6
Ai Xuan
6969b6df32f0333869ff94a7
Linqi Song
6969b6df32f0333869ff94a8
Jacky Keung
admin_assigned
2026-01-16T15:46:46.552Z
67c6ec64c60daca4757e5821
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/hXpsfp_QNxm4LQRkudPq-.png
Jacky Keung
jackykeung
user
jackykeung
LaViT: Aligning Latent Visual Thoughts for Multi-modal Reasoning
Current multimodal latent reasoning often relies on external supervision (e.g., auxiliary images), ignoring intrinsic visual attention dynamics. In this work, we identify a critical Perception Gap in distillation: student models frequently mimic a teacher's textual output while attending to fundamentally divergent visual regions, effectively relying on language priors rather than grounded perception. To bridge this, we propose LaViT, a framework that aligns latent visual thoughts rather than static embeddings. LaViT compels the student to autoregressively reconstruct the teacher's visual semantics and attention trajectories prior to text generation, employing a curriculum sensory gating mechanism to prevent shortcut learning. Extensive experiments show that LaViT significantly enhances visual grounding, achieving up to +16.9% gains on complex reasoning tasks and enabling a compact 3B model to outperform larger open-source variants and proprietary models like GPT-4o.
LaViT addresses the perception gap in multimodal reasoning by aligning latent visual thoughts through autoregressive reconstruction of visual semantics and attention trajectories, improving visual grounding and model performance.
multimodal latent reasoning external supervision intrinsic visual attention dynamics Perception Gap distillation student models teacher models textual output visual regions language priors grounded perception LaViT latent visual thoughts autoregressive reconstruction visual semantics attention trajectories curriculum sensory gating mechanism shortcut learning complex reasoning tasks visual grounding