Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models
Building general-purpose reasoning models with reinforcement learning (RL) entails substantial cross-domain heterogeneity, including large variation in inference-time response lengths and verification latency. Such variability complicates the RL infrastructure, slows training, and makes training curriculum (e.g., response length extension) and hyperparameter selection challenging. In this work, we propose cascaded domain-wise reinforcement learning (Cascade RL) to develop general-purpose reasoning models, Nemotron-Cascade, capable of operating in both instruct and deep thinking modes. Departing from conventional approaches that blend heterogeneous prompts from different domains, Cascade RL orchestrates sequential, domain-wise RL, reducing engineering complexity and delivering state-of-the-art performance across a wide range of benchmarks. Notably, RLHF for alignment, when used as a pre-step, boosts the model's reasoning ability far beyond mere preference optimization, and subsequent domain-wise RLVR stages rarely degrade the benchmark performance attained in earlier domains and may even improve it (see an illustration in Figure 1). Our 14B model, after RL, outperforms its SFT teacher, DeepSeek-R1-0528, on LiveCodeBench v5/v6/Pro and achieves silver-medal performance in the 2025 International Olympiad in Informatics (IOI). We transparently share our training and data recipes.
6940d34965f1e24a11780427
Boxin Wang
6940d34965f1e24a11780428
Chankyu Lee
6940d34965f1e24a11780429
Nayeon Lee
6940d34965f1e24a1178042a
Sheng-Chieh Lin
6940d34965f1e24a1178042b
Wenliang Dai
6940d34965f1e24a1178042c
Yang Chen
claimed_verified
2025-12-18T08:00:29.823Z
62bc9d90e81dfd65cced9316
/avatars/05df14cd1fdbc7d6a80d2960a05a94f0.svg
Yang Chen
ychenNLP
user
ychenNLP
6940d34965f1e24a1178042d
Yangyi Chen
claimed_verified
2025-12-19T09:07:50.838Z
64b70c63104e7af01cfcaca2
https://cdn-avatars.huggingface.co/v1/production/uploads/64b70c63104e7af01cfcaca2/iFJjigrViLQR1zX6Ll5ED.jpeg
Yangyi Chen
YangyiYY
user
YangyiYY
6940d34965f1e24a1178042e
Zhuolin Yang
claimed_verified
2025-12-18T08:00:31.420Z
67d75b0117c2acac528f47b6
/avatars/619aacd1a619aab64de3499ac3ee2229.svg
Zhuolin Yang
zhuoliny
user
zhuoliny
6940d34965f1e24a1178042f
Zihan Liu
6940d34965f1e24a11780430
Mohammad Shoeybi
6940d34965f1e24a11780431
Bryan Catanzaro
6940d34965f1e24a11780432
Wei Ping
Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models
Building general-purpose reasoning models with reinforcement learning (RL) entails substantial cross-domain heterogeneity, including large variation in inference-time response lengths and verification latency. Such variability complicates the RL infrastructure, slows training, and makes training curriculum (e.g., response length extension) and hyperparameter selection challenging. In this work, we propose cascaded domain-wise reinforcement learning (Cascade RL) to develop general-purpose reasoning models, Nemotron-Cascade, capable of operating in both instruct and deep thinking modes. Departing from conventional approaches that blend heterogeneous prompts from different domains, Cascade RL orchestrates sequential, domain-wise RL, reducing engineering complexity and delivering state-of-the-art performance across a wide range of benchmarks. Notably, RLHF for alignment, when used as a pre-step, boosts the model's reasoning ability far beyond mere preference optimization, and subsequent domain-wise RLVR stages rarely degrade the benchmark performance attained in earlier domains and may even improve it (see an illustration in Figure 1). Our 14B model, after RL, outperforms its SFT teacher, DeepSeek-R1-0528, on LiveCodeBench v5/v6/Pro and achieves silver-medal performance in the 2025 International Olympiad in Informatics (IOI). We transparently share our training and data recipes.
Cascaded domain-wise reinforcement learning (Cascade RL) is proposed to enhance general-purpose reasoning models, achieving state-of-the-art performance across benchmarks and outperforming the teacher model in coding competitions.
reinforcement learning RL Cascade RL Nemotron-Cascade instruct mode deep thinking mode RLHF alignment RLVR LiveCodeBench International Olympiad in Informatics (IOI)