Reinforced Attention Learning
Post-training with Reinforcement Learning (RL) has substantially improved reasoning in Large Language Models (LLMs) via test-time scaling. However, extending this paradigm to Multimodal LLMs (MLLMs) through verbose rationales yields limited gains for perception and can even degrade performance.
  We propose Reinforced Attention Learning (RAL), a policy-gradient framework that directly optimizes internal attention distributions rather than output token sequences. By shifting optimization from what to generate to where to attend, RAL promotes effective information allocation and improved grounding in complex multimodal inputs. Experiments across diverse image and video benchmarks show consistent gains over GRPO and other baselines. We further introduce On-Policy Attention Distillation, demonstrating that transferring latent attention behaviors yields stronger cross-modal alignment than standard knowledge distillation. Our results position attention policies as a principled and general alternative for multimodal post-training.
698565d64ad556f294b7ebee
Bangzheng Li
698565d64ad556f294b7ebef
Jianmo Ni
698565d64ad556f294b7ebf0
Chen Qu
698565d64ad556f294b7ebf1
Ian Miao
698565d64ad556f294b7ebf2
Liu Yang
698565d64ad556f294b7ebf3
Xingyu Fu
claimed_verified
2026-02-09T08:35:32.418Z
6336091b2db86a181ccd6054
/avatars/829f69436225d05d2c2136bc90f640d7.svg
Xingyu Fu
Fiaa
user
Fiaa
698565d64ad556f294b7ebf4
Muhao Chen
698565d64ad556f294b7ebf5
Derek Zhiyuan Cheng
Reinforced Attention Learning
Post-training with Reinforcement Learning (RL) has substantially improved reasoning in Large Language Models (LLMs) via test-time scaling. However, extending this paradigm to Multimodal LLMs (MLLMs) through verbose rationales yields limited gains for perception and can even degrade performance.
  We propose Reinforced Attention Learning (RAL), a policy-gradient framework that directly optimizes internal attention distributions rather than output token sequences. By shifting optimization from what to generate to where to attend, RAL promotes effective information allocation and improved grounding in complex multimodal inputs. Experiments across diverse image and video benchmarks show consistent gains over GRPO and other baselines. We further introduce On-Policy Attention Distillation, demonstrating that transferring latent attention behaviors yields stronger cross-modal alignment than standard knowledge distillation. Our results position attention policies as a principled and general alternative for multimodal post-training.
Reinforced Attention Learning optimizes internal attention distributions in multimodal language models, improving information allocation and cross-modal alignment through policy-gradient methods.
Reinforcement Learning Large Language Models Multimodal LLMs policy-gradient framework attention distributions test-time scaling GRPO On-Policy Attention Distillation cross-modal alignment knowledge distillation