LK Losses: Direct Acceptance Rate Optimization for Speculative Decoding
Speculative decoding accelerates autoregressive large language model (LLM) inference by using a lightweight draft model to propose candidate tokens that are then verified in parallel by the target model. The speedup is significantly determined by the acceptance rate, yet standard training minimizes Kullback-Leibler (KL) divergence as a proxy objective. While KL divergence and acceptance rate share the same global optimum, small draft models, having limited capacity, typically converge to suboptimal solutions where minimizing KL does not guarantee maximizing acceptance rate. To address this issue, we propose LK losses, special training objectives that directly target acceptance rate. Comprehensive experiments across four draft architectures and six target models, ranging from 8B to 685B parameters, demonstrate consistent improvements in acceptance metrics across all configurations compared to the standard KL-based training. We evaluate our approach on general, coding and math domains and report gains of up to 8-10% in average acceptance length. LK losses are easy to implement, introduce no computational overhead and can be directly integrated into any existing speculator training framework, making them a compelling alternative to the existing draft training objectives.
69a55931bc2d66e68888f640
Alexander Samarin
claimed_verified
2026-03-03T10:42:19.442Z
67ed1803ea852d670331cfd8
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/Q-hHk4q-ENMKCA8mPg3zH.png
Aleksandr Samarin
astrlrd
user
astrlrd
69a55931bc2d66e68888f641
Sergei Krutikov
69a55931bc2d66e68888f642
Anton Shevtsov
claimed_verified
2026-03-03T10:42:22.272Z
66214f06d55e5f677e0da9f2
/avatars/dc3b8de4a72b0d10f6bb90594488a9ec.svg
Anton
mrshevan
user
mrshevan
69a55931bc2d66e68888f643
Sergei Skvortsov
claimed_verified
2026-03-02T10:56:28.266Z
648467389c368e41d2a26e0b
/avatars/3d5cd7e9be776f4f3b134de6d5c75af5.svg
Sergei
southfreebird
user
southfreebird
69a55931bc2d66e68888f644
Filipp Fisin
69a55931bc2d66e68888f645
Alexander Golubev
claimed_verified
2026-03-02T09:45:29.072Z
644e9ffcd6001776ed77d874
/avatars/b93e02caf929679b7e9bc589eed0b689.svg
Alexander
djalexj
user
djalexj
LK Losses: Direct Acceptance Rate Optimization for Speculative Decoding
Speculative decoding accelerates autoregressive large language model (LLM) inference by using a lightweight draft model to propose candidate tokens that are then verified in parallel by the target model. The speedup is significantly determined by the acceptance rate, yet standard training minimizes Kullback-Leibler (KL) divergence as a proxy objective. While KL divergence and acceptance rate share the same global optimum, small draft models, having limited capacity, typically converge to suboptimal solutions where minimizing KL does not guarantee maximizing acceptance rate. To address this issue, we propose LK losses, special training objectives that directly target acceptance rate. Comprehensive experiments across four draft architectures and six target models, ranging from 8B to 685B parameters, demonstrate consistent improvements in acceptance metrics across all configurations compared to the standard KL-based training. We evaluate our approach on general, coding and math domains and report gains of up to 8-10% in average acceptance length. LK losses are easy to implement, introduce no computational overhead and can be directly integrated into any existing speculator training framework, making them a compelling alternative to the existing draft training objectives.
Speculative decoding speedup is improved by LK losses that directly optimize acceptance rate instead of KL divergence, showing consistent performance gains across different model sizes and domains.
speculative decoding autoregressive large language model draft model candidate tokens target model acceptance rate Kullback-Leibler divergence LK losses training objectives model capacity