On-Policy Self-Distillation for Reasoning Compression
Reasoning models think out loud, but much of what they say is noise. We introduce OPSDC (On-Policy Self-Distillation for Reasoning Compression), a method that teaches models to reason more concisely by
  distilling their own concise behavior back into themselves. The entire approach reduces to one idea: condition the same model on a "be concise" instruction to obtain teacher logits, and minimize per-token
  reverse KL on the student's own rollouts. No ground-truth answers, no token budgets, no difficulty estimators. Just self-distillation. Yet this simplicity belies surprising sophistication: OPSDC automatically
  compresses easy problems aggressively while preserving the deliberation needed for hard ones. On Qwen3-8B and Qwen3-14B, we achieve 57-59% token reduction on MATH-500 while improving accuracy by 9-16 points
  absolute. On AIME 2024, the 14B model gains 10 points with 41% compression. The secret? Much of what reasoning models produce is not just redundant-it is actively harmful, compounding errors with every
  unnecessary token.
69aa4ad4c4f6edddcc6c8687
Hejian Sang
69aa4ad4c4f6edddcc6c8688
Yuanda Xu
claimed_verified
2026-03-13T08:15:06.993Z
69a2f97d2927b6e8151f8563
/avatars/96bf509fd2e3b90cd339801e8b9b33cb.svg
XYX
xuyd16
user
xuyd16
69aa4ad4c4f6edddcc6c8689
Zhengze Zhou
69aa4ad4c4f6edddcc6c868a
Ran He
69aa4ad4c4f6edddcc6c868b
Zhipeng Wang
69aa4ad4c4f6edddcc6c868c
Jiachen Sun
On-Policy Self-Distillation for Reasoning Compression
Reasoning models think out loud, but much of what they say is noise. We introduce OPSDC (On-Policy Self-Distillation for Reasoning Compression), a method that teaches models to reason more concisely by
  distilling their own concise behavior back into themselves. The entire approach reduces to one idea: condition the same model on a "be concise" instruction to obtain teacher logits, and minimize per-token
  reverse KL on the student's own rollouts. No ground-truth answers, no token budgets, no difficulty estimators. Just self-distillation. Yet this simplicity belies surprising sophistication: OPSDC automatically
  compresses easy problems aggressively while preserving the deliberation needed for hard ones. On Qwen3-8B and Qwen3-14B, we achieve 57-59% token reduction on MATH-500 while improving accuracy by 9-16 points
  absolute. On AIME 2024, the 14B model gains 10 points with 41% compression. The secret? Much of what reasoning models produce is not just redundant-it is actively harmful, compounding errors with every
  unnecessary token.
OPSDC enables efficient reasoning model compression by having models distill concise behavior from their own outputs, achieving significant token reduction while maintaining accuracy.
self-distillation reverse KL on-policy reasoning compression teacher logits student rollouts token reduction MATH-500 AIME 2024