ProFit: Leveraging High-Value Signals in SFT via Probability-Guided Token Selection
Supervised fine-tuning (SFT) is a fundamental post-training strategy to align Large Language Models (LLMs) with human intent. However, traditional SFT often ignores the one-to-many nature of language by forcing alignment with a single reference answer, leading to the model overfitting to non-core expressions. Although our empirical analysis suggests that introducing multiple reference answers can mitigate this issue, the prohibitive data and computational costs necessitate a strategic shift: prioritizing the mitigation of single-reference overfitting over the costly pursuit of answer diversity. To achieve this, we reveal the intrinsic connection between token probability and semantic importance: high-probability tokens carry the core logical framework, while low-probability tokens are mostly replaceable expressions. Based on this insight, we propose ProFit, which selectively masks low-probability tokens to prevent surface-level overfitting. Extensive experiments confirm that ProFit consistently outperforms traditional SFT baselines on general reasoning and mathematical benchmarks.
6969ca6832f0333869ff94c0
Tao Liu
claimed_verified
2026-01-19T14:54:40.157Z
67be83a727ba0c1993f48c4f
https://cdn-avatars.huggingface.co/v1/production/uploads/67be83a727ba0c1993f48c4f/0FFHsI_S-htU4a9PfHLLA.jpeg
Tao Liu
utaotao
user
utaotao
6969ca6832f0333869ff94c1
Taiqiang Wu
claimed_verified
2026-01-19T09:23:59.501Z
6621cea88850e38ffbb1854f
https://cdn-avatars.huggingface.co/v1/production/uploads/6621cea88850e38ffbb1854f/qmKHts7YCUylxqzPHk4uT.jpeg
Taki WU
taki555
user
taki555
6969ca6832f0333869ff94c2
Runming Yang
claimed_verified
2026-01-19T14:33:54.695Z
66441bbd6df04abec508648e
/avatars/dcbc33742318d357ab9d426d12efa89a.svg
Rummy
yang31210999
user
yang31210999
6969ca6832f0333869ff94c3
Shaoning Sun
admin_assigned
2026-01-19T14:41:33.930Z
656af95af7be0986b44e7eef
/avatars/8b0c25ddb1d248a2eed0928f3403d521.svg
Shaoning Sun
shaoningsun
user
shaoningsun
6969ca6832f0333869ff94c4
Junjie Wang
claimed_verified
2026-01-20T09:41:17.098Z
62579c55b98dcaa7e0de285d
https://cdn-avatars.huggingface.co/v1/production/uploads/62579c55b98dcaa7e0de285d/0YUd5nloul_bW9yolDGGo.jpeg
wangjunjie
wanng
user
wanng
6969ca6832f0333869ff94c5
Yujiu Yang
admin_assigned
2026-01-19T14:41:43.285Z
64ca1fe838837b12d5e529b7
/avatars/44a3ad9e59318784ac531993b5f69f6b.svg
Yujiu Yang
Thu-redrobot
user
Thu-redrobot
ProFit: Leveraging High-Value Signals in SFT via Probability-Guided Token Selection
Supervised fine-tuning (SFT) is a fundamental post-training strategy to align Large Language Models (LLMs) with human intent. However, traditional SFT often ignores the one-to-many nature of language by forcing alignment with a single reference answer, leading to the model overfitting to non-core expressions. Although our empirical analysis suggests that introducing multiple reference answers can mitigate this issue, the prohibitive data and computational costs necessitate a strategic shift: prioritizing the mitigation of single-reference overfitting over the costly pursuit of answer diversity. To achieve this, we reveal the intrinsic connection between token probability and semantic importance: high-probability tokens carry the core logical framework, while low-probability tokens are mostly replaceable expressions. Based on this insight, we propose ProFit, which selectively masks low-probability tokens to prevent surface-level overfitting. Extensive experiments confirm that ProFit consistently outperforms traditional SFT baselines on general reasoning and mathematical benchmarks.
Supervised fine-tuning with multiple references addresses overfitting to non-core expressions by masking low-probability tokens based on their semantic importance.
supervised fine-tuning Large Language Models one-to-many nature token probability semantic importance ProFit surface-level overfitting