The AI Hippocampus: How Far are We From Human Memory?
Memory plays a foundational role in augmenting the reasoning, adaptability, and contextual fidelity of modern Large Language Models and Multi-Modal LLMs. As these models transition from static predictors to interactive systems capable of continual learning and personalized inference, the incorporation of memory mechanisms has emerged as a central theme in their architectural and functional evolution. This survey presents a comprehensive and structured synthesis of memory in LLMs and MLLMs, organizing the literature into a cohesive taxonomy comprising implicit, explicit, and agentic memory paradigms. Specifically, the survey delineates three primary memory frameworks. Implicit memory refers to the knowledge embedded within the internal parameters of pre-trained transformers, encompassing their capacity for memorization, associative retrieval, and contextual reasoning. Recent work has explored methods to interpret, manipulate, and reconfigure this latent memory. Explicit memory involves external storage and retrieval components designed to augment model outputs with dynamic, queryable knowledge representations, such as textual corpora, dense vectors, and graph-based structures, thereby enabling scalable and updatable interaction with information sources. Agentic memory introduces persistent, temporally extended memory structures within autonomous agents, facilitating long-term planning, self-consistency, and collaborative behavior in multi-agent systems, with relevance to embodied and interactive AI. Extending beyond text, the survey examines the integration of memory within multi-modal settings, where coherence across vision, language, audio, and action modalities is essential. Key architectural advances, benchmark tasks, and open challenges are discussed, including issues related to memory capacity, alignment, factual consistency, and cross-system interoperability.
696867330ac10a06522f6a71
Zixia Jia
696867330ac10a06522f6a72
Jiaqi Li
696867330ac10a06522f6a73
Yipeng Kang
696867330ac10a06522f6a74
Yuxuan Wang
696867330ac10a06522f6a75
Tong Wu
696867330ac10a06522f6a76
Quansen Wang
696867330ac10a06522f6a77
Xiaobo Wang
claimed_verified
2026-03-09T10:40:55.803Z
65e933fcacb29a596cd6eb6e
https://cdn-avatars.huggingface.co/v1/production/uploads/65e933fcacb29a596cd6eb6e/dlH14rO2I4EUMVyjEjdRK.jpeg
Xiaobo Wang
Yofuria
user
Yofuria
696867330ac10a06522f6a78
Shuyi Zhang
696867330ac10a06522f6a79
Junzhe Shen
696867330ac10a06522f6a7a
Qing Li
696867330ac10a06522f6a7b
Siyuan Qi
696867330ac10a06522f6a7c
Yitao Liang
696867330ac10a06522f6a7d
Di He
696867330ac10a06522f6a7e
Zilong Zheng
696867330ac10a06522f6a7f
Song-Chun Zhu
The AI Hippocampus: How Far are We From Human Memory?
Memory plays a foundational role in augmenting the reasoning, adaptability, and contextual fidelity of modern Large Language Models and Multi-Modal LLMs. As these models transition from static predictors to interactive systems capable of continual learning and personalized inference, the incorporation of memory mechanisms has emerged as a central theme in their architectural and functional evolution. This survey presents a comprehensive and structured synthesis of memory in LLMs and MLLMs, organizing the literature into a cohesive taxonomy comprising implicit, explicit, and agentic memory paradigms. Specifically, the survey delineates three primary memory frameworks. Implicit memory refers to the knowledge embedded within the internal parameters of pre-trained transformers, encompassing their capacity for memorization, associative retrieval, and contextual reasoning. Recent work has explored methods to interpret, manipulate, and reconfigure this latent memory. Explicit memory involves external storage and retrieval components designed to augment model outputs with dynamic, queryable knowledge representations, such as textual corpora, dense vectors, and graph-based structures, thereby enabling scalable and updatable interaction with information sources. Agentic memory introduces persistent, temporally extended memory structures within autonomous agents, facilitating long-term planning, self-consistency, and collaborative behavior in multi-agent systems, with relevance to embodied and interactive AI. Extending beyond text, the survey examines the integration of memory within multi-modal settings, where coherence across vision, language, audio, and action modalities is essential. Key architectural advances, benchmark tasks, and open challenges are discussed, including issues related to memory capacity, alignment, factual consistency, and cross-system interoperability.
Memory mechanisms in large language models and multi-modal language models are categorized into implicit, explicit, and agentic paradigms, supporting enhanced reasoning, adaptability, and contextual fidelity through internal parameters, external knowledge storage, and persistent agent memory structures.
Large Language Models Multi-Modal LLMs memory mechanisms implicit memory explicit memory agentic memory transformers memorization associative retrieval contextual reasoning external storage dynamic knowledge representations persistent memory structures long-term planning self-consistency multi-agent systems embodied AI interactive AI memory capacity factual consistency cross-system interoperability