Shorter but not Worse: Frugal Reasoning via Easy Samples as Length
  Regularizers in Math RLVR
Large language models (LLMs) trained for step-by-step reasoning often become
excessively verbose, raising inference cost. Standard Reinforcement Learning
with Verifiable Rewards (RLVR) pipelines filter out ``easy'' problems for
training efficiency, leaving the model to train primarily on harder problems
that require longer reasoning chains. This skews the output length distribution
upward, resulting in a model that conflates ``thinking longer'' with
``thinking better''. In this work, we show that retaining and modestly
up-weighting moderately easy problems acts as an implicit length regularizer.
Exposing the model to solvable short-chain tasks constrains its output
distribution and prevents runaway verbosity. The result is
\emph{emergent brevity for free}: the model learns to solve harder
problems without inflating the output length,  despite the absence of
any explicit length penalization. RLVR experiments using this approach on
Qwen3-4B-Thinking-2507 (with a 16k token limit) achieve baseline
pass@1 AIME25 accuracy while generating solutions that are, on average, nearly
twice as short. The code is available at
https://github.com/MBZUAI-Paris/Frugal-AI{GitHub}, with datasets and
models on
https://huggingface.co/collections/MBZUAI-Paris/k2-think-mini-68dcfa8b114686a4bd3dc2bc{Hugging
Face}.
690ae1c8d70e173c84529059
Abdelaziz Bounhar
claimed_verified
2025-11-05T16:24:33.298Z
6380e53efb49cd1c12052c17
https://cdn-avatars.huggingface.co/v1/production/uploads/6380e53efb49cd1c12052c17/b5CweexfrVn-W_xto2agR.jpeg
Abdelaziz Bounhar
BounharAbdelaziz
user
BounharAbdelaziz
690ae1c8d70e173c8452905a
Hadi Abdine
690ae1c8d70e173c8452905b
Evan Dufraisse
690ae1c8d70e173c8452905c
Ahmad Chamma
claimed_verified
2025-12-15T08:14:42.076Z
6751b0caecaa275e389dd5eb
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/RI2Y_nnjg5xJnv5sJVx_v.png
Ahmad Chamma
AC-723
user
AC-723
690ae1c8d70e173c8452905d
Amr Mohamed
690ae1c8d70e173c8452905e
Dani Bouch
690ae1c8d70e173c8452905f
Michalis Vazirgiannis
690ae1c8d70e173c84529060
Guokan Shang
claimed_verified
2025-11-05T16:24:31.211Z
6087e598e2b7cc3a117b0dc5
https://cdn-avatars.huggingface.co/v1/production/uploads/6087e598e2b7cc3a117b0dc5/Ctz_W-uo1gOQRBHXalD1P.png
Guokan Shang
guokan-shang
user
guokan-shang
Shorter but not Worse: Frugal Reasoning via Easy Samples as Length
  Regularizers in Math RLVR
Large language models (LLMs) trained for step-by-step reasoning often become
excessively verbose, raising inference cost. Standard Reinforcement Learning
with Verifiable Rewards (RLVR) pipelines filter out ``easy'' problems for
training efficiency, leaving the model to train primarily on harder problems
that require longer reasoning chains. This skews the output length distribution
upward, resulting in a model that conflates ``thinking longer'' with
``thinking better''. In this work, we show that retaining and modestly
up-weighting moderately easy problems acts as an implicit length regularizer.
Exposing the model to solvable short-chain tasks constrains its output
distribution and prevents runaway verbosity. The result is
\emph{emergent brevity for free}: the model learns to solve harder
problems without inflating the output length,  despite the absence of
any explicit length penalization. RLVR experiments using this approach on
Qwen3-4B-Thinking-2507 (with a 16k token limit) achieve baseline
pass@1 AIME25 accuracy while generating solutions that are, on average, nearly
twice as short. The code is available at
https://github.com/MBZUAI-Paris/Frugal-AI{GitHub}, with datasets and
models on
https://huggingface.co/collections/MBZUAI-Paris/k2-think-mini-68dcfa8b114686a4bd3dc2bc{Hugging
Face}.
Retaining and up-weighting moderately easy problems in RLVR pipelines for LLMs reduces output verbosity without explicit length penalization.
Reinforcement Learning with Verifiable Rewards (RLVR) Large language models (LLMs) step-by-step reasoning inference cost output length distribution thinking longer thinking better emergent brevity Qwen3-4B-Thinking-2507 pass@1 AIME25 accuracy