The End of Manual Decoding: Towards Truly End-to-End Language Models
The "end-to-end" label for LLMs is a misnomer. In practice, they depend on a
non-differentiable decoding process that requires laborious, hand-tuning of
hyperparameters like temperature and top-p. This paper introduces AutoDeco, a
novel architecture that enables truly "end-to-end" generation by learning to
control its own decoding strategy. We augment the standard transformer with
lightweight heads that, at each step, dynamically predict context-specific
temperature and top-p values alongside the next-token logits. This approach
transforms decoding into a parametric, token-level process, allowing the model
to self-regulate its sampling strategy within a single forward pass.
  Through extensive experiments on eight benchmarks, we demonstrate that
AutoDeco not only significantly outperforms default decoding strategies but
also achieves performance comparable to an oracle-tuned baseline derived from
"hacking the test set"-a practical upper bound for any static method.
Crucially, we uncover an emergent capability for instruction-based decoding
control: the model learns to interpret natural language commands (e.g.,
"generate with low randomness") and adjusts its predicted temperature and top-p
on a token-by-token basis, opening a new paradigm for steerable and interactive
LLM decoding.
690466b52c556835fa67f16e
Zhichao Wang
claimed_verified
2025-10-31T14:26:18.863Z
6645e248b03e00c6a0f61b88
/avatars/97af419819044ff90e1937916d6377b1.svg
zhichaowang
zacks917
user
zacks917
690466b52c556835fa67f16f
Dongyang Ma
claimed_verified
2025-10-31T14:26:15.874Z
6734a0fe3ed65dd196e40cfa
/avatars/dfa0f7e8d53ff611d3af51d3c10adac6.svg
madongyang
ldsjmdy
user
ldsjmdy
690466b52c556835fa67f170
Xinting Huang
claimed_verified
2025-11-03T20:54:43.838Z
63c227a6c58fcfeac18dec07
/avatars/47df92c937cfaf8d726c7e28cf352eb5.svg
tim huang
timmyhhh
user
timmyhhh
690466b52c556835fa67f171
Deng Cai
690466b52c556835fa67f172
Tian Lan
claimed_verified
2025-10-31T14:26:21.582Z
627ca439a09edfe62239c671
https://cdn-avatars.huggingface.co/v1/production/uploads/1652335660508-noauth.jpeg
Tian Lan
GMFTBY
user
GMFTBY
690466b52c556835fa67f173
Jiahao Xu
690466b52c556835fa67f174
Haitao Mi
690466b52c556835fa67f175
Xiaoying Tang
claimed_verified
2025-11-03T20:54:38.111Z
69087cf1b8c21a27dbad5fee
/avatars/4ac66747570cd89c64e1560235604927.svg
XiaoyingTANG
XiaoyingTANG
user
XiaoyingTANG
690466b52c556835fa67f176
Yan Wang
claimed_verified
2025-11-03T20:54:41.057Z
64674375946476c5d215382d
/avatars/0f0b98080c064a17d003da8a22a047cc.svg
Yan Wang
libertywing
user
libertywing
The End of Manual Decoding: Towards Truly End-to-End Language Models
The "end-to-end" label for LLMs is a misnomer. In practice, they depend on a
non-differentiable decoding process that requires laborious, hand-tuning of
hyperparameters like temperature and top-p. This paper introduces AutoDeco, a
novel architecture that enables truly "end-to-end" generation by learning to
control its own decoding strategy. We augment the standard transformer with
lightweight heads that, at each step, dynamically predict context-specific
temperature and top-p values alongside the next-token logits. This approach
transforms decoding into a parametric, token-level process, allowing the model
to self-regulate its sampling strategy within a single forward pass.
  Through extensive experiments on eight benchmarks, we demonstrate that
AutoDeco not only significantly outperforms default decoding strategies but
also achieves performance comparable to an oracle-tuned baseline derived from
"hacking the test set"-a practical upper bound for any static method.
Crucially, we uncover an emergent capability for instruction-based decoding
control: the model learns to interpret natural language commands (e.g.,
"generate with low randomness") and adjusts its predicted temperature and top-p
on a token-by-token basis, opening a new paradigm for steerable and interactive
LLM decoding.
AutoDeco, a novel architecture, enables end-to-end generation by dynamically predicting decoding parameters, outperforming default strategies and achieving performance close to an oracle-tuned baseline.
LLMs end-to-end decoding process hyperparameters temperature top-p AutoDeco transformer lightweight heads next-token logits parametric token-level process self-regulation sampling strategy instruction-based decoding control natural language commands