CoBA-RL: Capability-Oriented Budget Allocation for Reinforcement Learning in LLMs
Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a key approach for enhancing LLM reasoning.However, standard frameworks like Group Relative Policy Optimization (GRPO) typically employ a uniform rollout budget, leading to resource inefficiency. Moreover, existing adaptive methods often rely on instance-level metrics, such as task pass rates, failing to capture the model's dynamic learning state. To address these limitations, we propose CoBA-RL, a reinforcement learning algorithm designed to adaptively allocate rollout budgets based on the model's evolving capability. Specifically, CoBA-RL utilizes a Capability-Oriented Value function to map tasks to their potential training gains and employs a heap-based greedy strategy to efficiently self-calibrate the distribution of computational resources to samples with high training value. Extensive experiments demonstrate that our approach effectively orchestrates the trade-off between exploration and exploitation, delivering consistent generalization improvements across multiple challenging benchmarks. These findings underscore that quantifying sample training value and optimizing budget allocation are pivotal for advancing LLM post-training efficiency.
6982c1039084cb4f0ecb572c
Zhiyuan Yao
claimed_verified
2026-02-04T12:28:25.743Z
6708edcae69f6e30a816af9f
/avatars/c4daa9b0cb2f4bb2a7db0e78b22034cb.svg
Yao
distant-yuan
user
distant-yuan
6982c1039084cb4f0ecb572d
Yi-Kai Zhang
6982c1039084cb4f0ecb572e
Yuxin Chen
6982c1039084cb4f0ecb572f
Yueqing Sun
6982c1039084cb4f0ecb5730
Zishan Xu
6982c1039084cb4f0ecb5731
Yu Yang
6982c1039084cb4f0ecb5732
Tianhao Hu
6982c1039084cb4f0ecb5733
Qi Gu
6982c1039084cb4f0ecb5734
Hui Su
6982c1039084cb4f0ecb5735
Xunliang Cai
CoBA-RL: Capability-Oriented Budget Allocation for Reinforcement Learning in LLMs
Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a key approach for enhancing LLM reasoning.However, standard frameworks like Group Relative Policy Optimization (GRPO) typically employ a uniform rollout budget, leading to resource inefficiency. Moreover, existing adaptive methods often rely on instance-level metrics, such as task pass rates, failing to capture the model's dynamic learning state. To address these limitations, we propose CoBA-RL, a reinforcement learning algorithm designed to adaptively allocate rollout budgets based on the model's evolving capability. Specifically, CoBA-RL utilizes a Capability-Oriented Value function to map tasks to their potential training gains and employs a heap-based greedy strategy to efficiently self-calibrate the distribution of computational resources to samples with high training value. Extensive experiments demonstrate that our approach effectively orchestrates the trade-off between exploration and exploitation, delivering consistent generalization improvements across multiple challenging benchmarks. These findings underscore that quantifying sample training value and optimizing budget allocation are pivotal for advancing LLM post-training efficiency.
CoBA-RL adapts rollout budget allocation for LLM training by evaluating sample training value and optimizing resource distribution through a capability-oriented value function and greedy strategy.
Reinforcement Learning with Verifiable Rewards GRPO rollout budget adaptive methods Capability-Oriented Value function heap-based greedy strategy exploration and exploitation LLM post-training efficiency