Nemotron Elastic: Towards Efficient Many-in-One Reasoning LLMs
Training a family of large language models targeting multiple scales and deployment objectives is prohibitively expensive, requiring separate training runs for each different size. Recent work on model compression through pruning and knowledge distillation has reduced this cost; however, this process still incurs hundreds of billions of tokens worth of training cost per compressed model. In this paper, we present Nemotron Elastic, a framework for building reasoning-oriented LLMs, including hybrid Mamba-Attention architectures, that embed multiple nested submodels within a single parent model, each optimized for different deployment configurations and budgets. Each of these submodels shares weights with the parent model and can be extracted zero-shot during deployment without additional training or fine-tuning. We enable this functionality through an end-to-end trained router, tightly coupled to a two-stage training curriculum designed specifically for reasoning models. We additionally introduce group-aware SSM elastification that preserves Mamba's structural constraints, heterogeneous MLP elastification, normalized MSE-based layer importance for improved depth selection, and knowledge distillation enabling simultaneous multi-budget optimization. We apply Nemotron Elastic to the Nemotron Nano V2 12B model, simultaneously producing a 9B and a 6B model using only 110B training tokens; this results in over 360x cost reduction compared to training model families from scratch, and around 7x compared to SoTA compression techniques. Each of the nested models performs on par or better than the SoTA in accuracy. Moreover, unlike other compression methods, the nested capability of our approach allows having a many-in-one reasoning model that has constant deployment memory against the number of models in the family.
691fcf18cce0eb9b6387aef9
Ali Taghibakhshi
691fcf18cce0eb9b6387aefa
Sharath Turuvekere Sreenivas
691fcf18cce0eb9b6387aefb
Saurav Muralidharan
691fcf18cce0eb9b6387aefc
Ruisi Cai
691fcf18cce0eb9b6387aefd
Marcin Chochowski
691fcf18cce0eb9b6387aefe
Ameya Sunil Mahabaleshwarkar
691fcf18cce0eb9b6387aeff
Yoshi Suhara
691fcf18cce0eb9b6387af00
Oluwatobi Olabiyi
691fcf18cce0eb9b6387af01
Daniel Korzekwa
691fcf18cce0eb9b6387af02
Mostofa Patwary
691fcf18cce0eb9b6387af03
Mohammad Shoeybi
691fcf18cce0eb9b6387af04
Jan Kautz
691fcf18cce0eb9b6387af05
Bryan Catanzaro
691fcf18cce0eb9b6387af06
Ashwath Aithal
691fcf18cce0eb9b6387af07
Nima Tajbakhsh
691fcf18cce0eb9b6387af08
Pavlo Molchanov
claimed_verified
2026-01-22T08:48:03.425Z
646d0c1c534e52f8c30500a6
https://cdn-avatars.huggingface.co/v1/production/uploads/646d0c1c534e52f8c30500a6/75VH8ClbRaP75BU2ONfXE.png
Pavlo Molchanov
pmolchanov
user
pmolchanov
Nemotron Elastic: Towards Efficient Many-in-One Reasoning LLMs
Training a family of large language models targeting multiple scales and deployment objectives is prohibitively expensive, requiring separate training runs for each different size. Recent work on model compression through pruning and knowledge distillation has reduced this cost; however, this process still incurs hundreds of billions of tokens worth of training cost per compressed model. In this paper, we present Nemotron Elastic, a framework for building reasoning-oriented LLMs, including hybrid Mamba-Attention architectures, that embed multiple nested submodels within a single parent model, each optimized for different deployment configurations and budgets. Each of these submodels shares weights with the parent model and can be extracted zero-shot during deployment without additional training or fine-tuning. We enable this functionality through an end-to-end trained router, tightly coupled to a two-stage training curriculum designed specifically for reasoning models. We additionally introduce group-aware SSM elastification that preserves Mamba's structural constraints, heterogeneous MLP elastification, normalized MSE-based layer importance for improved depth selection, and knowledge distillation enabling simultaneous multi-budget optimization. We apply Nemotron Elastic to the Nemotron Nano V2 12B model, simultaneously producing a 9B and a 6B model using only 110B training tokens; this results in over 360x cost reduction compared to training model families from scratch, and around 7x compared to SoTA compression techniques. Each of the nested models performs on par or better than the SoTA in accuracy. Moreover, unlike other compression methods, the nested capability of our approach allows having a many-in-one reasoning model that has constant deployment memory against the number of models in the family.
Nemotron Elastic reduces training costs and memory usage by embedding multiple submodels within a single large language model, optimized for various deployment configurations and budgets without additional training or fine-tuning.
large language models model compression pruning knowledge distillation hybrid Mamba-Attention architectures nested submodels parent model zero-shot extraction end-to-end trained router two-stage training curriculum group-aware SSM elastification heterogeneous MLP elastification normalized MSE-based layer importance multi-budget optimization Nemotron Nano V2 12B model