Latent Chain-of-Thought for Visual Reasoning
Chain-of-thought (CoT) reasoning is critical for improving the
interpretability and reliability of Large Vision-Language Models (LVLMs).
However, existing training algorithms such as SFT, PPO, and GRPO may not
generalize well across unseen reasoning tasks and heavily rely on a biased
reward model. To address this challenge, we reformulate reasoning in LVLMs as
posterior inference and propose a scalable training algorithm based on
amortized variational inference. By leveraging diversity-seeking reinforcement
learning algorithms, we introduce a novel sparse reward function for
token-level learning signals that encourage diverse, high-likelihood latent
CoT, overcoming deterministic sampling limitations and avoiding reward hacking.
Additionally, we implement a Bayesian inference-scaling strategy that replaces
costly Best-of-N and Beam Search with a marginal likelihood to efficiently rank
optimal rationales and answers. We empirically demonstrate that the proposed
method enhances the state-of-the-art LVLMs on seven reasoning benchmarks, in
terms of effectiveness, generalization, and interpretability.
69022e74646208eac0d1f726
Guohao Sun
69022e74646208eac0d1f727
Hang Hua
claimed_verified
2025-10-30T14:39:37.479Z
639f8277beb95d698de007dd
/avatars/57f223ccd9d3cb03166ccf0e41361c58.svg
HangHua
hhua2
user
hhua2
69022e74646208eac0d1f728
Jian Wang
69022e74646208eac0d1f729
Jiebo Luo
69022e74646208eac0d1f72a
Sohail Dianat
69022e74646208eac0d1f72b
Majid Rabbani
69022e74646208eac0d1f72c
Raghuveer Rao
69022e74646208eac0d1f72d
Zhiqiang Tao
Latent Chain-of-Thought for Visual Reasoning
Chain-of-thought (CoT) reasoning is critical for improving the
interpretability and reliability of Large Vision-Language Models (LVLMs).
However, existing training algorithms such as SFT, PPO, and GRPO may not
generalize well across unseen reasoning tasks and heavily rely on a biased
reward model. To address this challenge, we reformulate reasoning in LVLMs as
posterior inference and propose a scalable training algorithm based on
amortized variational inference. By leveraging diversity-seeking reinforcement
learning algorithms, we introduce a novel sparse reward function for
token-level learning signals that encourage diverse, high-likelihood latent
CoT, overcoming deterministic sampling limitations and avoiding reward hacking.
Additionally, we implement a Bayesian inference-scaling strategy that replaces
costly Best-of-N and Beam Search with a marginal likelihood to efficiently rank
optimal rationales and answers. We empirically demonstrate that the proposed
method enhances the state-of-the-art LVLMs on seven reasoning benchmarks, in
terms of effectiveness, generalization, and interpretability.
The proposed method reformulates reasoning in Large Vision-Language Models as posterior inference using amortized variational inference and a sparse reward function, improving effectiveness, generalization, and interpretability.
Chain-of-thought Large Vision-Language Models SFT PPO GRPO posterior inference amortized variational inference diversity-seeking reinforcement learning sparse reward function token-level learning signals latent CoT deterministic sampling reward hacking Bayesian inference-scaling strategy marginal likelihood reasoning benchmarks