Beyond Reasoning Gains: Mitigating General Capabilities Forgetting in
  Large Reasoning Models
Reinforcement learning with verifiable rewards (RLVR) has delivered
impressive gains in mathematical and multimodal reasoning and has become a
standard post-training paradigm for contemporary language and vision-language
models. However, the RLVR recipe introduces a significant risk of capability
regression, where models forget foundational skills after prolonged training
without employing regularization strategies. We empirically confirm this
concern, observing that open-source reasoning models suffer performance
degradation on core capabilities such as perception and faithfulness. While
imposing regularization terms like KL divergence can help prevent deviation
from the base model, these terms are calculated on the current task, thus they
do not guarantee broader knowledge. Meanwhile, commonly used experience replay
across heterogeneous domains makes it nontrivial to decide how much training
focus each objective should receive. To address this, we propose RECAP-a replay
strategy with dynamic objective reweighting for general knowledge preservation.
Our reweighting mechanism adapts in an online manner using short-horizon
signals of convergence and instability, shifting the post-training focus away
from saturated objectives and toward underperforming or volatile ones. Our
method is end-to-end and readily applicable to existing RLVR pipelines without
training additional models or heavy tuning. Extensive experiments on benchmarks
based on Qwen2.5-VL-3B and Qwen2.5-VL-7B demonstrate the effectiveness of our
method, which not only preserves general capabilities but also improves
reasoning by enabling more flexible trade-offs among in-task rewards.
6900dd3f646208eac0d1f072
Hoang Phan
6900dd3f646208eac0d1f073
Xianjun Yang
6900dd3f646208eac0d1f074
Kevin Yao
6900dd3f646208eac0d1f075
Jingyu Zhang
claimed_verified
2026-04-15T12:10:47.304Z
62fb40b59af1d16bc0ac60f4
/avatars/03ff66a419db8f2bc8e89a3b47aaaeac.svg
Jack Zhang
jackzhang
user
jackzhang
6900dd3f646208eac0d1f076
Shengjie Bi
6900dd3f646208eac0d1f077
Xiaocheng Tang
6900dd3f646208eac0d1f078
Madian Khabsa
6900dd3f646208eac0d1f079
Lijuan Liu
6900dd3f646208eac0d1f07a
Deren Lei
claimed_verified
2025-10-29T12:49:28.128Z
66c3c0cbdb33653b9041f968
https://cdn-avatars.huggingface.co/v1/production/uploads/66c3c0cbdb33653b9041f968/U2DEh4uyvhKnz7fNNyhyc.png
Deren Lei
derenlei
user
derenlei
Beyond Reasoning Gains: Mitigating General Capabilities Forgetting in
  Large Reasoning Models
Reinforcement learning with verifiable rewards (RLVR) has delivered
impressive gains in mathematical and multimodal reasoning and has become a
standard post-training paradigm for contemporary language and vision-language
models. However, the RLVR recipe introduces a significant risk of capability
regression, where models forget foundational skills after prolonged training
without employing regularization strategies. We empirically confirm this
concern, observing that open-source reasoning models suffer performance
degradation on core capabilities such as perception and faithfulness. While
imposing regularization terms like KL divergence can help prevent deviation
from the base model, these terms are calculated on the current task, thus they
do not guarantee broader knowledge. Meanwhile, commonly used experience replay
across heterogeneous domains makes it nontrivial to decide how much training
focus each objective should receive. To address this, we propose RECAP-a replay
strategy with dynamic objective reweighting for general knowledge preservation.
Our reweighting mechanism adapts in an online manner using short-horizon
signals of convergence and instability, shifting the post-training focus away
from saturated objectives and toward underperforming or volatile ones. Our
method is end-to-end and readily applicable to existing RLVR pipelines without
training additional models or heavy tuning. Extensive experiments on benchmarks
based on Qwen2.5-VL-3B and Qwen2.5-VL-7B demonstrate the effectiveness of our
method, which not only preserves general capabilities but also improves
reasoning by enabling more flexible trade-offs among in-task rewards.
RECAP, a dynamic objective reweighting strategy, enhances reinforcement learning with verifiable rewards by preserving general knowledge and improving reasoning through flexible reward trade-offs.
reinforcement learning verifiable rewards capability regression regularization KL divergence experience replay dynamic objective reweighting short-horizon signals convergence instability Qwen2.5-VL-3B Qwen2.5-VL-7B