JustRL: Scaling a 1.5B LLM with a Simple RL Recipe
Recent advances in reinforcement learning for large language models have converged on increasing complexity: multi-stage training pipelines, dynamic hyperparameter schedules, and curriculum learning strategies. This raises a fundamental question: Is this complexity necessary? We present JustRL, a minimal approach using single-stage training with fixed hyperparameters that achieves state-of-the-art performance on two 1.5B reasoning models (54.9\% and 64.3\% average accuracy across nine mathematical benchmarks) while using 2times less compute than sophisticated approaches. The same hyperparameters transfer across both models without tuning, and training exhibits smooth, monotonic improvement over 4,000+ steps without the collapses or plateaus that typically motivate interventions. Critically, ablations reveal that adding ``standard tricks'' like explicit length penalties and robust verifiers may degrade performance by collapsing exploration. These results suggest that the field may be adding complexity to solve problems that disappear with a stable, scaled-up baseline. We release our models and code to establish a simple, validated baseline for the community.
6944b6f3fbf17e708e185f4c
Bingxiang He
6944b6f3fbf17e708e185f4d
Zekai Qu
6944b6f3fbf17e708e185f4e
Zeyuan Liu
6944b6f3fbf17e708e185f4f
Yinghao Chen
6944b6f3fbf17e708e185f50
Yuxin Zuo
claimed_verified
2026-04-15T12:10:28.983Z
622474f38dc6b0b64f5e903d
/avatars/d6b60a014277a8ec7d564163c5f644aa.svg
Yuxin Zuo
yuxinzuo
user
yuxinzuo
6944b6f3fbf17e708e185f51
Cheng Qian
6944b6f3fbf17e708e185f52
Kaiyan Zhang
6944b6f3fbf17e708e185f53
Weize Chen
6944b6f3fbf17e708e185f54
Chaojun Xiao
6944b6f3fbf17e708e185f55
Ganqu Cui
6944b6f3fbf17e708e185f56
Ning Ding
6944b6f3fbf17e708e185f57
Zhiyuan Liu
JustRL: Scaling a 1.5B LLM with a Simple RL Recipe
Recent advances in reinforcement learning for large language models have converged on increasing complexity: multi-stage training pipelines, dynamic hyperparameter schedules, and curriculum learning strategies. This raises a fundamental question: Is this complexity necessary? We present JustRL, a minimal approach using single-stage training with fixed hyperparameters that achieves state-of-the-art performance on two 1.5B reasoning models (54.9\% and 64.3\% average accuracy across nine mathematical benchmarks) while using 2times less compute than sophisticated approaches. The same hyperparameters transfer across both models without tuning, and training exhibits smooth, monotonic improvement over 4,000+ steps without the collapses or plateaus that typically motivate interventions. Critically, ablations reveal that adding ``standard tricks'' like explicit length penalties and robust verifiers may degrade performance by collapsing exploration. These results suggest that the field may be adding complexity to solve problems that disappear with a stable, scaled-up baseline. We release our models and code to establish a simple, validated baseline for the community.
JustRL achieves state-of-the-art performance on reasoning models with minimal complexity, using single-stage training and fixed hyperparameters, outperforming sophisticated approaches in terms of compute and stability.
reinforcement learning large language models multi-stage training dynamic hyperparameter schedules curriculum learning single-stage training fixed hyperparameters reasoning models mathematical benchmarks explicit length penalties robust verifiers stable baseline