BrowseComp-V^3: A Visual, Vertical, and Verifiable Benchmark for Multimodal Browsing Agents
Multimodal large language models (MLLMs), equipped with increasingly advanced planning and tool-use capabilities, are evolving into autonomous agents capable of performing multimodal web browsing and deep search in open-world environments. However, existing benchmarks for multimodal browsing remain limited in task complexity, evidence accessibility, and evaluation granularity, hindering comprehensive and reproducible assessments of deep search capabilities. To address these limitations, we introduce BrowseComp-V^3, a novel benchmark consisting of 300 carefully curated and challenging questions spanning diverse domains. The benchmark emphasizes deep, multi-level, and cross-modal multi-hop reasoning, where critical evidence is interleaved across textual and visual modalities within and across web pages. All supporting evidence is strictly required to be publicly searchable, ensuring fairness and reproducibility. Beyond final-answer accuracy, we incorporate an expert-validated, subgoal-driven process evaluation mechanism that enables fine-grained analysis of intermediate reasoning behaviors and systematic characterization of capability boundaries. In addition, we propose OmniSeeker, a unified multimodal browsing agent framework integrating diverse web search and visual perception tools. Comprehensive experiments demonstrate that even state-of-the-art models achieve only 36% accuracy on our benchmark, revealing critical bottlenecks in multimodal information integration and fine-grained perception. Our results highlight a fundamental gap between current model capabilities and robust multimodal deep search in real-world settings.
6993e3db50fb2c0be4783d2d
Huanyao Zhang
6993e3db50fb2c0be4783d2e
Jiepeng Zhou
6993e3db50fb2c0be4783d2f
Bo Li
6993e3db50fb2c0be4783d30
Bowen Zhou
6993e3db50fb2c0be4783d31
Yanzhe Dan
6993e3db50fb2c0be4783d32
Haishan Lu
6993e3db50fb2c0be4783d33
Zhiyong Cao
6993e3db50fb2c0be4783d34
Jiaoyang Chen
6993e3db50fb2c0be4783d35
Yuqian Han
6993e3db50fb2c0be4783d36
Zinan Sheng
6993e3db50fb2c0be4783d37
Zhengwei Tao
6993e3db50fb2c0be4783d38
Hao Liang
claimed_verified
2026-04-07T08:48:55.602Z
6751a4fedf636b0140a9b873
/avatars/d75f7f6cfbfb4d646e0e557d1cfacdce.svg
Hao Liang
lhpku20010120
user
lhpku20010120
6993e3db50fb2c0be4783d39
Jialong Wu
claimed_verified
2026-02-20T08:37:43.360Z
644a4fbc2166258fccc664bc
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/8k3b44MbhQiWuo6i8BnYl.jpeg
Jialong Wu
callanwu
user
callanwu
6993e3db50fb2c0be4783d3a
Yang Shi
claimed_verified
2026-02-18T09:07:01.848Z
673c7319d11b1c2e246ead9c
https://cdn-avatars.huggingface.co/v1/production/uploads/673c7319d11b1c2e246ead9c/IjFIO--N7Hm_BOEafhEQv.jpeg
Yang Shi
DogNeverSleep
user
DogNeverSleep
6993e3db50fb2c0be4783d3b
Yuanpeng He
6993e3db50fb2c0be4783d3c
Jiaye Lin
6993e3db50fb2c0be4783d3d
Qintong Zhang
6993e3db50fb2c0be4783d3e
Guochen Yan
6993e3db50fb2c0be4783d3f
Runhao Zhao
6993e3db50fb2c0be4783d40
Zhengpin Li
6993e3db50fb2c0be4783d41
Xiaohan Yu
6993e3db50fb2c0be4783d42
Lang Mei
6993e3db50fb2c0be4783d43
Chong Chen
6993e3db50fb2c0be4783d44
Wentao Zhang
6993e3db50fb2c0be4783d45
Bin Cui
BrowseComp-V^3: A Visual, Vertical, and Verifiable Benchmark for Multimodal Browsing Agents
Multimodal large language models (MLLMs), equipped with increasingly advanced planning and tool-use capabilities, are evolving into autonomous agents capable of performing multimodal web browsing and deep search in open-world environments. However, existing benchmarks for multimodal browsing remain limited in task complexity, evidence accessibility, and evaluation granularity, hindering comprehensive and reproducible assessments of deep search capabilities. To address these limitations, we introduce BrowseComp-V^3, a novel benchmark consisting of 300 carefully curated and challenging questions spanning diverse domains. The benchmark emphasizes deep, multi-level, and cross-modal multi-hop reasoning, where critical evidence is interleaved across textual and visual modalities within and across web pages. All supporting evidence is strictly required to be publicly searchable, ensuring fairness and reproducibility. Beyond final-answer accuracy, we incorporate an expert-validated, subgoal-driven process evaluation mechanism that enables fine-grained analysis of intermediate reasoning behaviors and systematic characterization of capability boundaries. In addition, we propose OmniSeeker, a unified multimodal browsing agent framework integrating diverse web search and visual perception tools. Comprehensive experiments demonstrate that even state-of-the-art models achieve only 36% accuracy on our benchmark, revealing critical bottlenecks in multimodal information integration and fine-grained perception. Our results highlight a fundamental gap between current model capabilities and robust multimodal deep search in real-world settings.
A new benchmark called BrowseComp-V3 challenges multimodal large language models with complex, multi-hop reasoning tasks requiring deep search across text and visual modalities, revealing significant gaps in current capabilities.
multimodal large language models multimodal browsing deep search web browsing multimodal information integration fine-grained perception multimodal browsing agent framework web search visual perception