HippoCamp: Benchmarking Contextual Agents on Personal Computers
We present HippoCamp, a new benchmark designed to evaluate agents' capabilities on multimodal file management. Unlike existing agent benchmarks that focus on tasks like web interaction, tool use, or software automation in generic settings, HippoCamp evaluates agents in user-centric environments to model individual user profiles and search massive personal files for context-aware reasoning. Our benchmark instantiates device-scale file systems over real-world profiles spanning diverse modalities, comprising 42.4 GB of data across over 2K real-world files. Building upon the raw files, we construct 581 QA pairs to assess agents' capabilities in search, evidence perception, and multi-step reasoning. To facilitate fine-grained analysis, we provide 46.1K densely annotated structured trajectories for step-wise failure diagnosis. We evaluate a wide range of state-of-the-art multimodal large language models (MLLMs) and agentic methods on HippoCamp. Our comprehensive experiments reveal a significant performance gap: even the most advanced commercial models achieve only 48.3% accuracy in user profiling, struggling particularly with long-horizon retrieval and cross-modal reasoning within dense personal file systems. Furthermore, our step-wise failure diagnosis identifies multimodal perception and evidence grounding as the primary bottlenecks. Ultimately, HippoCamp exposes the critical limitations of current agents in realistic, user-centric environments and provides a robust foundation for developing next-generation personal AI assistants.
69cdcdbaacf91530b21df90d
Zhe Yang
claimed_verified
2026-04-02T08:14:45.435Z
66c7e8d021fc3eab824eab70
https://cdn-avatars.huggingface.co/v1/production/uploads/66c7e8d021fc3eab824eab70/WbU1ILNEOoLNMYWhsHtG5.jpeg
YANG Zhe
Savannah-yz
user
Savannah-yz
69cdcdbaacf91530b21df90e
Shulin Tian
69cdcdbaacf91530b21df90f
Kairui Hu
69cdcdbaacf91530b21df910
Shuai Liu
claimed_verified
2026-04-02T15:31:43.315Z
64f7f5b54101c731ca84ae05
https://cdn-avatars.huggingface.co/v1/production/uploads/64f7f5b54101c731ca84ae05/13DwdxOo3tWbxKDLd44B9.jpeg
Shuai Liu
Choiszt
user
Choiszt
69cdcdbaacf91530b21df911
Hoang-Nhat Nguyen
69cdcdbaacf91530b21df912
Yichi Zhang
claimed_verified
2026-04-02T08:14:42.965Z
66bb3e4b4de574a3298f8f64
/avatars/6eff3ec8016b8f2a59182b5f046248c5.svg
Catill Zhang
Catill520
user
Catill520
69cdcdbaacf91530b21df913
Zujin Guo
69cdcdbaacf91530b21df914
Mengying Yu
69cdcdbaacf91530b21df915
Zinan Zhang
69cdcdbaacf91530b21df916
Jingkang Yang
claimed_verified
2026-04-07T08:46:40.636Z
62b5777f593a2c49da69dc02
https://cdn-avatars.huggingface.co/v1/production/uploads/1658152070753-62b5777f593a2c49da69dc02.jpeg
Jingkang Yang
Jingkang
user
Jingkang
69cdcdbaacf91530b21df917
Chen Change Loy
69cdcdbaacf91530b21df918
Ziwei Liu
HippoCamp: Benchmarking Contextual Agents on Personal Computers
We present HippoCamp, a new benchmark designed to evaluate agents' capabilities on multimodal file management. Unlike existing agent benchmarks that focus on tasks like web interaction, tool use, or software automation in generic settings, HippoCamp evaluates agents in user-centric environments to model individual user profiles and search massive personal files for context-aware reasoning. Our benchmark instantiates device-scale file systems over real-world profiles spanning diverse modalities, comprising 42.4 GB of data across over 2K real-world files. Building upon the raw files, we construct 581 QA pairs to assess agents' capabilities in search, evidence perception, and multi-step reasoning. To facilitate fine-grained analysis, we provide 46.1K densely annotated structured trajectories for step-wise failure diagnosis. We evaluate a wide range of state-of-the-art multimodal large language models (MLLMs) and agentic methods on HippoCamp. Our comprehensive experiments reveal a significant performance gap: even the most advanced commercial models achieve only 48.3% accuracy in user profiling, struggling particularly with long-horizon retrieval and cross-modal reasoning within dense personal file systems. Furthermore, our step-wise failure diagnosis identifies multimodal perception and evidence grounding as the primary bottlenecks. Ultimately, HippoCamp exposes the critical limitations of current agents in realistic, user-centric environments and provides a robust foundation for developing next-generation personal AI assistants.
HippoCamp is a multimodal file management benchmark that evaluates agents' capabilities in user-centric environments, revealing significant performance gaps in long-horizon retrieval and cross-modal reasoning within dense personal file systems.
multimodal large language models agentic methods user profiling long-horizon retrieval cross-modal reasoning evidence grounding