Anchored Decoding: Provably Reducing Copyright Risk for Any Language Model
Modern language models (LMs) tend to memorize portions of their training data and emit verbatim spans. When the underlying sources are sensitive or copyright-protected, such reproduction raises issues of consent and compensation for creators and compliance risks for developers. We propose Anchored Decoding, a plug-and-play inference-time method for suppressing verbatim copying: it enables decoding from any risky LM trained on mixed-license data by keeping generation in bounded proximity to a permissively trained safe LM. Anchored Decoding adaptively allocates a user-chosen information budget over the generation trajectory and enforces per-step constraints that yield a sequence-level guarantee, enabling a tunable risk-utility trade-off. To make Anchored Decoding practically useful, we introduce a new permissively trained safe model (TinyComma 1.8B), as well as Anchored_{Byte} Decoding, a byte-level variant of our method that enables cross-vocabulary fusion via the ByteSampler framework (Hayase et al., 2025). We evaluate our methods across six model pairs on long-form evaluations of copyright risk and utility. Anchored and Anchored_{Byte} Decoding define a new Pareto frontier, preserving near-original fluency and factuality while eliminating up to 75% of the measurable copying gap (averaged over six copying metrics) between the risky baseline and a safe reference, at a modest inference overhead.
698add0e1b2dc6b37d61b2b0
Jacqueline He
claimed_verified
2026-02-11T11:17:54.124Z
627c52e51b02d7b5bf671dca
https://cdn-avatars.huggingface.co/v1/production/uploads/1666240522285-627c52e51b02d7b5bf671dca.png
Jacqueline He
jacquelinehe
user
jacquelinehe
698add0e1b2dc6b37d61b2b1
Jonathan Hayase
698add0e1b2dc6b37d61b2b2
Wen-tau Yih
698add0e1b2dc6b37d61b2b3
Sewoong Oh
698add0e1b2dc6b37d61b2b4
Luke Zettlemoyer
698add0e1b2dc6b37d61b2b5
Pang Wei Koh
Anchored Decoding: Provably Reducing Copyright Risk for Any Language Model
Modern language models (LMs) tend to memorize portions of their training data and emit verbatim spans. When the underlying sources are sensitive or copyright-protected, such reproduction raises issues of consent and compensation for creators and compliance risks for developers. We propose Anchored Decoding, a plug-and-play inference-time method for suppressing verbatim copying: it enables decoding from any risky LM trained on mixed-license data by keeping generation in bounded proximity to a permissively trained safe LM. Anchored Decoding adaptively allocates a user-chosen information budget over the generation trajectory and enforces per-step constraints that yield a sequence-level guarantee, enabling a tunable risk-utility trade-off. To make Anchored Decoding practically useful, we introduce a new permissively trained safe model (TinyComma 1.8B), as well as Anchored_{Byte} Decoding, a byte-level variant of our method that enables cross-vocabulary fusion via the ByteSampler framework (Hayase et al., 2025). We evaluate our methods across six model pairs on long-form evaluations of copyright risk and utility. Anchored and Anchored_{Byte} Decoding define a new Pareto frontier, preserving near-original fluency and factuality while eliminating up to 75% of the measurable copying gap (averaged over six copying metrics) between the risky baseline and a safe reference, at a modest inference overhead.
Anchor decoding suppresses verbatim copying in language models while maintaining fluency and factual accuracy through constrained generation that balances risk and utility.
Anchored Decoding language models verbatim copying safe LM information budget per-step constraints Pareto frontier TinyComma 1.8B Anchored$_{\mathrm{Byte}}$ Decoding ByteSampler framework