DRIVE: Data Curation Best Practices for Reinforcement Learning with
  Verifiable Reward in Competitive Code Generation
Recent reasoning-first models (e.g., OpenAI o1, DeepSeek R1) have spurred a
resurgence of interest in RLVR. Nevertheless, advances are dominated by
mathematics (e.g., AIME), with competitive-programming code generation
underexplored and data curation receiving less attention than RL algorithm
design. We investigate how to construct RLVR datasets (i.e., RL prompts) and
present practical training techniques that yield strong performance on
competitive-programming code generation. Our pipeline begins with supervised
fine-tuning (SFT) distilled from strong open-source models, augmented with
general-purpose and reasoning-intensive data. RL then follows a two-stage
process with executable, testcase-driven rewards: first, training on a large,
uniformly distributed set of competitive-programming problems using Group
Relative Policy Optimization (GRPO) with 8 rollouts per prompt and a relatively
short response-generation window (e.g., 32k during SFT and 24k in this stage)
to expand entropy and mitigate repetition and truncation; second, we perform
Pre-GRPO: updating on a small, high-quality set of challenging
problems with a large rollout budget (64 rollouts per prompt) under a
hard-focus curriculum that continuously retains the most difficult instances
throughout training. We implement our method on Qwen2.5-32B and evaluate on
LeetCode and Codeforces weekly contests to avoid data leakage. The resulting
model achieves state-of-the-art performance among models of similar scale and
is comparable to leading systems such as DeepSeek v3.1 and Doubao-1.5-Thinking.
We also examine scaling trends and observe strong RL scaling on an internal
large-scale MoE model. Our study distills concise best practices for data
curation, entropy expansion, and curriculum design in RLVR for
competitive-programming code generation.
6912c5b6a644ba07c499c7ce
Speed Zhu
6912c5b6a644ba07c499c7cf
Jianwei Cai
6912c5b6a644ba07c499c7d0
Guang Chen
claimed_verified
2025-11-11T19:43:37.073Z
624445ed9fdefb55a0b9070b
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/MS8EoLxeEXZnBoXEbMEnn.png
GuangChen
Guang2666
user
Guang2666
6912c5b6a644ba07c499c7d1
Lulu Wu
6912c5b6a644ba07c499c7d2
Saiyong Yang
6912c5b6a644ba07c499c7d3
Wiggin Zhou
DRIVE: Data Curation Best Practices for Reinforcement Learning with
  Verifiable Reward in Competitive Code Generation
Recent reasoning-first models (e.g., OpenAI o1, DeepSeek R1) have spurred a
resurgence of interest in RLVR. Nevertheless, advances are dominated by
mathematics (e.g., AIME), with competitive-programming code generation
underexplored and data curation receiving less attention than RL algorithm
design. We investigate how to construct RLVR datasets (i.e., RL prompts) and
present practical training techniques that yield strong performance on
competitive-programming code generation. Our pipeline begins with supervised
fine-tuning (SFT) distilled from strong open-source models, augmented with
general-purpose and reasoning-intensive data. RL then follows a two-stage
process with executable, testcase-driven rewards: first, training on a large,
uniformly distributed set of competitive-programming problems using Group
Relative Policy Optimization (GRPO) with 8 rollouts per prompt and a relatively
short response-generation window (e.g., 32k during SFT and 24k in this stage)
to expand entropy and mitigate repetition and truncation; second, we perform
Pre-GRPO: updating on a small, high-quality set of challenging
problems with a large rollout budget (64 rollouts per prompt) under a
hard-focus curriculum that continuously retains the most difficult instances
throughout training. We implement our method on Qwen2.5-32B and evaluate on
LeetCode and Codeforces weekly contests to avoid data leakage. The resulting
model achieves state-of-the-art performance among models of similar scale and
is comparable to leading systems such as DeepSeek v3.1 and Doubao-1.5-Thinking.
We also examine scaling trends and observe strong RL scaling on an internal
large-scale MoE model. Our study distills concise best practices for data
curation, entropy expansion, and curriculum design in RLVR for
competitive-programming code generation.
The study presents a two-stage reinforcement learning approach for competitive-programming code generation, achieving state-of-the-art performance using Group Relative Policy Optimization and a hard-focus curriculum.
RLVR supervised fine-tuning Group Relative Policy Optimization Pre-GRPO entropy expansion curriculum design competitive-programming code generation Qwen2.5-32B LeetCode Codeforces MoE model