MemSifter: Offloading LLM Memory Retrieval via Outcome-Driven Proxy Reasoning
As Large Language Models (LLMs) are increasingly used for long-duration tasks, maintaining effective long-term memory has become a critical challenge. Current methods often face a trade-off between cost and accuracy. Simple storage methods often fail to retrieve relevant information, while complex indexing methods (such as memory graphs) require heavy computation and can cause information loss. Furthermore, relying on the working LLM to process all memories is computationally expensive and slow. To address these limitations, we propose MemSifter, a novel framework that offloads the memory retrieval process to a small-scale proxy model. Instead of increasing the burden on the primary working LLM, MemSifter uses a smaller model to reason about the task before retrieving the necessary information. This approach requires no heavy computation during the indexing phase and adds minimal overhead during inference. To optimize the proxy model, we introduce a memory-specific Reinforcement Learning (RL) training paradigm. We design a task-outcome-oriented reward based on the working LLM's actual performance in completing the task. The reward measures the actual contribution of retrieved memories by mutiple interactions with the working LLM, and discriminates retrieved rankings by stepped decreasing contributions. Additionally, we employ training techniques such as Curriculum Learning and Model Merging to improve performance. We evaluated MemSifter on eight LLM memory benchmarks, including Deep Research tasks. The results demonstrate that our method meets or exceeds the performance of existing state-of-the-art approaches in both retrieval accuracy and final task completion. MemSifter offers an efficient and scalable solution for long-term LLM memory. We have open-sourced the model weights, code, and training data to support further research.
69a8ffd18b5b27c7ea9fa66c
Jiejun Tan
claimed_verified
2026-03-05T10:01:59.169Z
62e52483a944e2a56cd2c6ca
https://cdn-avatars.huggingface.co/v1/production/uploads/62e52483a944e2a56cd2c6ca/xElM_6teIrP3QI-Run0Bl.jpeg
Jiejun Tan
zstanjj
user
zstanjj
69a8ffd18b5b27c7ea9fa66d
Zhicheng Dou
admin_assigned
2026-03-05T10:26:27.317Z
66f0bf59e9d50ec57febf751
/avatars/be97941e60064e5dd806c6fe9db3c537.svg
Zhicheng Dou
douzc
user
douzc
69a8ffd18b5b27c7ea9fa66e
Liancheng Zhang
69a8ffd18b5b27c7ea9fa66f
Yuyang Hu
claimed_verified
2026-03-06T13:56:42.768Z
6544b9b646dbdeca34ee5f52
https://cdn-avatars.huggingface.co/v1/production/uploads/6544b9b646dbdeca34ee5f52/nRx6m1C4wfZ_xSWoBUNJf.png
Yuyang Hu
namespace-ERI
user
namespace-ERI
69a8ffd18b5b27c7ea9fa670
Yiruo Cheng
admin_assigned
2026-03-05T10:26:47.129Z
6530eba73572cd4fb78daf36
/avatars/4e6f0f07bb4e0ed9e3c58d49eaf78fe5.svg
Yiruo Cheng
ariya2357
user
ariya2357
69a8ffd18b5b27c7ea9fa671
Ji-Rong Wen
admin_assigned
2026-03-05T10:26:53.023Z
64b8c89052b7353d8c6a1013
/avatars/cd59fffe81f6b07b4519540b8ff3d95f.svg
Ji-Rong Wen
jrwen
user
jrwen
MemSifter: Offloading LLM Memory Retrieval via Outcome-Driven Proxy Reasoning
As Large Language Models (LLMs) are increasingly used for long-duration tasks, maintaining effective long-term memory has become a critical challenge. Current methods often face a trade-off between cost and accuracy. Simple storage methods often fail to retrieve relevant information, while complex indexing methods (such as memory graphs) require heavy computation and can cause information loss. Furthermore, relying on the working LLM to process all memories is computationally expensive and slow. To address these limitations, we propose MemSifter, a novel framework that offloads the memory retrieval process to a small-scale proxy model. Instead of increasing the burden on the primary working LLM, MemSifter uses a smaller model to reason about the task before retrieving the necessary information. This approach requires no heavy computation during the indexing phase and adds minimal overhead during inference. To optimize the proxy model, we introduce a memory-specific Reinforcement Learning (RL) training paradigm. We design a task-outcome-oriented reward based on the working LLM's actual performance in completing the task. The reward measures the actual contribution of retrieved memories by mutiple interactions with the working LLM, and discriminates retrieved rankings by stepped decreasing contributions. Additionally, we employ training techniques such as Curriculum Learning and Model Merging to improve performance. We evaluated MemSifter on eight LLM memory benchmarks, including Deep Research tasks. The results demonstrate that our method meets or exceeds the performance of existing state-of-the-art approaches in both retrieval accuracy and final task completion. MemSifter offers an efficient and scalable solution for long-term LLM memory. We have open-sourced the model weights, code, and training data to support further research.
MemSifter is a framework that uses a small proxy model to offload memory retrieval from large language models, employing reinforcement learning with task-performance rewards and training techniques like curriculum learning and model merging to improve long-term memory efficiency and accuracy.
Large Language Models long-term memory memory retrieval proxy model Reinforcement Learning curriculum learning model merging task-outcome-oriented reward memory-specific RL training LLM memory benchmarks Deep Research tasks