Limits of Generalization in RLVR: Two Case Studies in Mathematical
  Reasoning
Mathematical reasoning is a central challenge for large language models
(LLMs), requiring not only correct answers but also faithful reasoning
processes. Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as
a promising approach for enhancing such capabilities; however, its ability to
foster genuine reasoning remains unclear. We investigate RLVR on two
combinatorial problems with fully verifiable solutions: Activity
Scheduling and the Longest Increasing Subsequence, using carefully
curated datasets with unique optima. Across multiple reward designs, we find
that RLVR improves evaluation metrics but often by reinforcing superficial
heuristics rather than acquiring new reasoning strategies. These findings
highlight the limits of RLVR generalization, emphasizing the importance of
benchmarks that disentangle genuine mathematical reasoning from shortcut
exploitation and provide faithful measures of progress. Code available at
https://github.com/xashru/rlvr-seq-generalization.
69087c52812eca10f9cc5f7e
Md Tanvirul Alam
claimed_verified
2025-11-17T10:32:17.530Z
620c1a977af55d45f5519914
/avatars/4d2fb7c1a5ad5dabdb8888fa2fe72e65.svg
Tanvirul Alam
xashru
user
xashru
69087c52812eca10f9cc5f7f
Nidhi Rastogi
Limits of Generalization in RLVR: Two Case Studies in Mathematical
  Reasoning
Mathematical reasoning is a central challenge for large language models
(LLMs), requiring not only correct answers but also faithful reasoning
processes. Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as
a promising approach for enhancing such capabilities; however, its ability to
foster genuine reasoning remains unclear. We investigate RLVR on two
combinatorial problems with fully verifiable solutions: Activity
Scheduling and the Longest Increasing Subsequence, using carefully
curated datasets with unique optima. Across multiple reward designs, we find
that RLVR improves evaluation metrics but often by reinforcing superficial
heuristics rather than acquiring new reasoning strategies. These findings
highlight the limits of RLVR generalization, emphasizing the importance of
benchmarks that disentangle genuine mathematical reasoning from shortcut
exploitation and provide faithful measures of progress. Code available at
https://github.com/xashru/rlvr-seq-generalization.
Reinforcement Learning with Verifiable Rewards (RLVR) enhances evaluation metrics on combinatorial problems but often reinforces superficial heuristics rather than genuine reasoning strategies.
Reinforcement Learning with Verifiable Rewards RLVR Activity Scheduling Longest Increasing Subsequence combinatorial problems verifiable solutions unique optima reward designs evaluation metrics superficial heuristics genuine reasoning strategies benchmarks shortcut exploitation faithful measures of progress