RLVE: Scaling Up Reinforcement Learning for Language Models with
  Adaptive Verifiable Environments
We introduce Reinforcement Learning (RL) with Adaptive Verifiable
Environments (RLVE), an approach using verifiable environments that
procedurally generate problems and provide algorithmically verifiable rewards,
to scale up RL for language models (LMs). RLVE enables each verifiable
environment to dynamically adapt its problem difficulty distribution to the
policy model's capabilities as training progresses. In contrast, static data
distributions often lead to vanishing learning signals when problems are either
too easy or too hard for the policy. To implement RLVE, we create RLVE-Gym, a
large-scale suite of 400 verifiable environments carefully developed through
manual environment engineering. Using RLVE-Gym, we show that environment
scaling, i.e., expanding the collection of training environments, consistently
improves generalizable reasoning capabilities. RLVE with joint training across
all 400 environments in RLVE-Gym yields a 3.37% absolute average improvement
across six reasoning benchmarks, starting from one of the strongest 1.5B
reasoning LMs. By comparison, continuing this LM's original RL training yields
only a 0.49% average absolute gain despite using over 3x more compute. We
release our code publicly.
6912bb2da644ba07c499c793
Zhiyuan Zeng
claimed_verified
2025-11-11T19:43:41.584Z
64a85e23b6512b8328f9d9e2
/avatars/4a6b35752d3f76cb03278f52b3b43426.svg
Zhiyuan Zeng
ZhiyuanZeng
user
ZhiyuanZeng
6912bb2da644ba07c499c794
Hamish Ivison
claimed_verified
2025-11-25T09:06:16.329Z
62608fc2ffe8827cb1d89f9f
https://cdn-avatars.huggingface.co/v1/production/uploads/1654027835241-62608fc2ffe8827cb1d89f9f.png
Hamish Ivison
hamishivi
user
hamishivi
6912bb2da644ba07c499c795
Yiping Wang
6912bb2da644ba07c499c796
Lifan Yuan
6912bb2da644ba07c499c797
Shuyue Stella Li
6912bb2da644ba07c499c798
Zhuorui Ye
6912bb2da644ba07c499c799
Siting Li
6912bb2da644ba07c499c79a
Jacqueline He
claimed_verified
2025-12-15T09:59:31.507Z
627c52e51b02d7b5bf671dca
https://cdn-avatars.huggingface.co/v1/production/uploads/1666240522285-627c52e51b02d7b5bf671dca.png
Jacqueline He
jacquelinehe
user
jacquelinehe
6912bb2da644ba07c499c79b
Runlong Zhou
6912bb2da644ba07c499c79c
Tong Chen
6912bb2da644ba07c499c79d
Chenyang Zhao
6912bb2da644ba07c499c79e
Yulia Tsvetkov
6912bb2da644ba07c499c79f
Simon Shaolei Du
6912bb2da644ba07c499c7a0
Natasha Jaques
6912bb2da644ba07c499c7a1
Hao Peng
6912bb2da644ba07c499c7a2
Pang Wei Koh
6912bb2da644ba07c499c7a3
Hannaneh Hajishirzi
RLVE: Scaling Up Reinforcement Learning for Language Models with
  Adaptive Verifiable Environments
We introduce Reinforcement Learning (RL) with Adaptive Verifiable
Environments (RLVE), an approach using verifiable environments that
procedurally generate problems and provide algorithmically verifiable rewards,
to scale up RL for language models (LMs). RLVE enables each verifiable
environment to dynamically adapt its problem difficulty distribution to the
policy model's capabilities as training progresses. In contrast, static data
distributions often lead to vanishing learning signals when problems are either
too easy or too hard for the policy. To implement RLVE, we create RLVE-Gym, a
large-scale suite of 400 verifiable environments carefully developed through
manual environment engineering. Using RLVE-Gym, we show that environment
scaling, i.e., expanding the collection of training environments, consistently
improves generalizable reasoning capabilities. RLVE with joint training across
all 400 environments in RLVE-Gym yields a 3.37% absolute average improvement
across six reasoning benchmarks, starting from one of the strongest 1.5B
reasoning LMs. By comparison, continuing this LM's original RL training yields
only a 0.49% average absolute gain despite using over 3x more compute. We
release our code publicly.
Reinforcement Learning with Adaptive Verifiable Environments (RLVE) improves language model reasoning by dynamically adjusting problem difficulty, outperforming static environments and traditional RL training.
Reinforcement Learning (RL) Adaptive Verifiable Environments (RLVE) verifiable environments procedural generation algorithmically verifiable rewards policy model problem difficulty distribution static data distributions vanishing learning signals RLVE-Gym environment scaling generalizable reasoning capabilities reasoning benchmarks language models (LMs)