GradMem: Learning to Write Context into Memory with Test-Time Gradient Descent
Many large language model applications require conditioning on long contexts. Transformers typically support this by storing a large per-layer KV-cache of past activations, which incurs substantial memory overhead. A desirable alternative is ompressive memory: read a context once, store it in a compact state, and answer many queries from that state. We study this in a context removal setting, where the model must generate an answer without access to the original context at inference time. We introduce GradMem, which writes context into memory via per-sample test-time optimization. Given a context, GradMem performs a few steps of gradient descent on a small set of prefix memory tokens while keeping model weights frozen. GradMem explicitly optimizes a model-level self-supervised context reconstruction loss, resulting in a loss-driven write operation with iterative error correction, unlike forward-only methods. On associative key--value retrieval, GradMem outperforms forward-only memory writers with the same memory size, and additional gradient steps scale capacity much more effectively than repeated forward writes. We further show that GradMem transfers beyond synthetic benchmarks: with pretrained language models, it attains competitive results on natural language tasks including bAbI and SQuAD variants, relying only on information encoded in memory.
69ba64635d2b9c28ea09f2ef
Yuri Kuratov
claimed_verified
2026-03-18T09:50:38.310Z
618b9540682ec1c38327e586
https://cdn-avatars.huggingface.co/v1/production/uploads/618b9540682ec1c38327e586/v_ZBkfh8O9Zh6C2YQpuBX.jpeg
Yury Kuratov
yurakuratov
user
yurakuratov
69ba64635d2b9c28ea09f2f0
Matvey Kairov
claimed_verified
2026-03-18T09:50:36.666Z
65e4a104cacb245553db1ee9
/avatars/0656d05242db6aedb268e64d27aad55d.svg
Matvey Kairov
mkairov
user
mkairov
69ba64635d2b9c28ea09f2f1
Aydar Bulatov
admin_assigned
2026-03-18T10:06:19.767Z
64c8b321cb2f1bf0e7c0f54b
https://cdn-avatars.huggingface.co/v1/production/uploads/64c8b321cb2f1bf0e7c0f54b/JflXxMVnG9I0IB5YNyhXF.jpeg
Aydar Bulatov
booydar
user
booydar
69ba64635d2b9c28ea09f2f2
Ivan Rodkin
admin_assigned
2026-03-18T10:06:37.916Z
63c1ac8cc58fcfeac186bda2
https://cdn-avatars.huggingface.co/v1/production/uploads/63c1ac8cc58fcfeac186bda2/MajMhLgsvCBN81tmO9PZc.jpeg
Ivan Rodkin
irodkin
user
irodkin
69ba64635d2b9c28ea09f2f3
Mikhail Burtsev
admin_assigned
2026-03-18T10:06:35.929Z
639c6e978a34ed9a404c6a7b
/avatars/c98ca8c9f9ed8509c2f1bb6aa994fd57.svg
MIKHAIL BURTSEV
mbur
user
mbur
GradMem: Learning to Write Context into Memory with Test-Time Gradient Descent
Many large language model applications require conditioning on long contexts. Transformers typically support this by storing a large per-layer KV-cache of past activations, which incurs substantial memory overhead. A desirable alternative is ompressive memory: read a context once, store it in a compact state, and answer many queries from that state. We study this in a context removal setting, where the model must generate an answer without access to the original context at inference time. We introduce GradMem, which writes context into memory via per-sample test-time optimization. Given a context, GradMem performs a few steps of gradient descent on a small set of prefix memory tokens while keeping model weights frozen. GradMem explicitly optimizes a model-level self-supervised context reconstruction loss, resulting in a loss-driven write operation with iterative error correction, unlike forward-only methods. On associative key--value retrieval, GradMem outperforms forward-only memory writers with the same memory size, and additional gradient steps scale capacity much more effectively than repeated forward writes. We further show that GradMem transfers beyond synthetic benchmarks: with pretrained language models, it attains competitive results on natural language tasks including bAbI and SQuAD variants, relying only on information encoded in memory.
GradMem enables efficient context storage and retrieval in language models through gradient-based memory writing that outperforms traditional forward-only approaches.
Transformers KV-cache compressive memory test-time optimization context reconstruction loss gradient descent memory tokens forward-only methods associative key-value retrieval language models