DeepImageSearch: Benchmarking Multimodal Agents for Context-Aware Image Retrieval in Visual Histories
Existing multimodal retrieval systems excel at semantic matching but implicitly assume that query-image relevance can be measured in isolation. This paradigm overlooks the rich dependencies inherent in realistic visual streams, where information is distributed across temporal sequences rather than confined to single snapshots. To bridge this gap, we introduce DeepImageSearch, a novel agentic paradigm that reformulates image retrieval as an autonomous exploration task. Models must plan and perform multi-step reasoning over raw visual histories to locate targets based on implicit contextual cues. We construct DISBench, a challenging benchmark built on interconnected visual data. To address the scalability challenge of creating context-dependent queries, we propose a human-model collaborative pipeline that employs vision-language models to mine latent spatiotemporal associations, effectively offloading intensive context discovery before human verification. Furthermore, we build a robust baseline using a modular agent framework equipped with fine-grained tools and a dual-memory system for long-horizon navigation. Extensive experiments demonstrate that DISBench poses significant challenges to state-of-the-art models, highlighting the necessity of incorporating agentic reasoning into next-generation retrieval systems.
698d8a1b65c0d15a6d1622db
Chenlong Deng
claimed_verified
2026-02-13T09:37:24.735Z
654c99d6e82a71cb487c2ecd
https://cdn-avatars.huggingface.co/v1/production/uploads/654c99d6e82a71cb487c2ecd/hiMMOyh-3bAUaqnBM5yT4.jpeg
ChenlongDeng
ChenlongDeng
user
ChenlongDeng
698d8a1b65c0d15a6d1622dc
Mengjie Deng
698d8a1b65c0d15a6d1622dd
Junjie Wu
698d8a1b65c0d15a6d1622de
Dun Zeng
698d8a1b65c0d15a6d1622df
Teng Wang
698d8a1b65c0d15a6d1622e0
Qingsong Xie
698d8a1b65c0d15a6d1622e1
Jiadeng Huang
claimed_verified
2026-02-17T15:50:53.691Z
6994899fe85a4b61cbc65813
/avatars/2b0fce4d53c59eb06ef0b023485ed81f.svg
Jiadeng Huang
Warden-H
user
Warden-H
698d8a1b65c0d15a6d1622e2
Shengjie Ma
698d8a1b65c0d15a6d1622e3
Changwang Zhang
claimed_verified
2026-04-13T07:38:47.044Z
67ea7f597382053ae1ff676f
https://cdn-avatars.huggingface.co/v1/production/uploads/67ea7f597382053ae1ff676f/zy6b17SFRBfxZvTnq5m3Z.jpeg
Changwang ZHANG
mLeoKing
user
mLeoKing
698d8a1b65c0d15a6d1622e4
Zhaoxiang Wang
698d8a1b65c0d15a6d1622e5
Jun Wang
698d8a1b65c0d15a6d1622e6
Yutao Zhu
698d8a1b65c0d15a6d1622e7
Zhicheng Dou
DeepImageSearch: Benchmarking Multimodal Agents for Context-Aware Image Retrieval in Visual Histories
Existing multimodal retrieval systems excel at semantic matching but implicitly assume that query-image relevance can be measured in isolation. This paradigm overlooks the rich dependencies inherent in realistic visual streams, where information is distributed across temporal sequences rather than confined to single snapshots. To bridge this gap, we introduce DeepImageSearch, a novel agentic paradigm that reformulates image retrieval as an autonomous exploration task. Models must plan and perform multi-step reasoning over raw visual histories to locate targets based on implicit contextual cues. We construct DISBench, a challenging benchmark built on interconnected visual data. To address the scalability challenge of creating context-dependent queries, we propose a human-model collaborative pipeline that employs vision-language models to mine latent spatiotemporal associations, effectively offloading intensive context discovery before human verification. Furthermore, we build a robust baseline using a modular agent framework equipped with fine-grained tools and a dual-memory system for long-horizon navigation. Extensive experiments demonstrate that DISBench poses significant challenges to state-of-the-art models, highlighting the necessity of incorporating agentic reasoning into next-generation retrieval systems.
DeepImageSearch presents an agentic approach to image retrieval that addresses limitations of traditional semantic matching by enabling multi-step reasoning over visual histories through a modular agent framework with dual-memory system.
multimodal retrieval systems visual streams temporal sequences agentic paradigm image retrieval visual history contextual cues DISBench vision-language models spatiotemporal associations modular agent framework dual-memory system long-horizon navigation