Which Reasoning Trajectories Teach Students to Reason Better? A Simple Metric of Informative Alignment
Long chain-of-thought (CoT) trajectories provide rich supervision signals for distilling reasoning from teacher to student LLMs. However, both prior work and our experiments show that trajectories from stronger teachers do not necessarily yield better students, highlighting the importance of data-student suitability in distillation. Existing methods assess suitability primarily through student likelihood, favoring trajectories that closely align with the model's current behavior but overlooking more informative ones. Addressing this, we propose Rank-Surprisal Ratio (RSR), a simple metric that captures both alignment and informativeness to assess the suitability of a reasoning trajectory. RSR is motivated by the observation that effective trajectories typically combine low absolute probability with relatively high-ranked tokens under the student model, balancing learning signal strength and behavioral alignment. Concretely, RSR is defined as the ratio of a trajectory's average token-wise rank to its average negative log-likelihood, and is straightforward to compute and interpret. Across five student models and reasoning trajectories from 11 diverse teachers, RSR strongly correlates with post-training performance (average Spearman 0.86), outperforming existing metrics. We further demonstrate its practical utility in both trajectory selection and teacher selection.
6970e682572de08d9c7ae914
Yuming Yang
claimed_verified
2026-01-23T20:15:23.750Z
655c6b1abfb531437a54c0e6
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/V8Md2mMX83hrSowKk6qMS.jpeg
Yuming Yang
Umean
user
Umean
6970e682572de08d9c7ae915
Mingyoung Lai
6970e682572de08d9c7ae916
Wanxu Zhao
6970e682572de08d9c7ae917
Xiaoran Fan
6970e682572de08d9c7ae918
Zhiheng Xi
6970e682572de08d9c7ae919
Mingqi Wu
6970e682572de08d9c7ae91a
Chiyue Huang
6970e682572de08d9c7ae91b
Jun Zhao
6970e682572de08d9c7ae91c
Haijun Lv
6970e682572de08d9c7ae91d
Jian Tong
6970e682572de08d9c7ae91e
Yunhua Zhou
6970e682572de08d9c7ae91f
Yicheng Zou
6970e682572de08d9c7ae920
Qipeng Guo
6970e682572de08d9c7ae921
Tao Gui
6970e682572de08d9c7ae922
Qi Zhang
6970e682572de08d9c7ae923
Xuanjing Huang
Which Reasoning Trajectories Teach Students to Reason Better? A Simple Metric of Informative Alignment
Long chain-of-thought (CoT) trajectories provide rich supervision signals for distilling reasoning from teacher to student LLMs. However, both prior work and our experiments show that trajectories from stronger teachers do not necessarily yield better students, highlighting the importance of data-student suitability in distillation. Existing methods assess suitability primarily through student likelihood, favoring trajectories that closely align with the model's current behavior but overlooking more informative ones. Addressing this, we propose Rank-Surprisal Ratio (RSR), a simple metric that captures both alignment and informativeness to assess the suitability of a reasoning trajectory. RSR is motivated by the observation that effective trajectories typically combine low absolute probability with relatively high-ranked tokens under the student model, balancing learning signal strength and behavioral alignment. Concretely, RSR is defined as the ratio of a trajectory's average token-wise rank to its average negative log-likelihood, and is straightforward to compute and interpret. Across five student models and reasoning trajectories from 11 diverse teachers, RSR strongly correlates with post-training performance (average Spearman 0.86), outperforming existing metrics. We further demonstrate its practical utility in both trajectory selection and teacher selection.
Researchers introduce a novel metric called Rank-Surprisal Ratio (RSR) to better assess the suitability of reasoning trajectories for distilling knowledge from large language models, demonstrating superior performance compared to existing methods.
reasoning trajectories distillation teacher-student LLMs student likelihood token-wise rank negative log-likelihood Spearman correlation