MMDeepResearch-Bench: A Benchmark for Multimodal Deep Research Agents
Deep Research Agents (DRAs) generate citation-rich reports via multi-step search and synthesis, yet existing benchmarks mainly target text-only settings or short-form multimodal QA, missing end-to-end multimodal evidence use. We introduce MMDeepResearch-Bench (MMDR-Bench), a benchmark of 140 expert-crafted tasks across 21 domains, where each task provides an image-text bundle to evaluate multimodal understanding and citation-grounded report generation. Compared to prior setups, MMDR-Bench emphasizes report-style synthesis with explicit evidence use, where models must connect visual artifacts to sourced claims and maintain consistency across narrative, citations, and visual references. We further propose a unified, interpretable evaluation pipeline: Formula-LLM Adaptive Evaluation (FLAE) for report quality, Trustworthy Retrieval-Aligned Citation Evaluation (TRACE) for citation-grounded evidence alignment, and Multimodal Support-Aligned Integrity Check (MOSAIC) for text-visual integrity, each producing fine-grained signals that support error diagnosis beyond a single overall score. Experiments across 25 state-of-the-art models reveal systematic trade-offs between generation quality, citation discipline, and multimodal grounding, highlighting that strong prose alone does not guarantee faithful evidence use and that multimodal integrity remains a key bottleneck for deep research agents.
697145b5c1c7409747bf94c7
Peizhou Huang
claimed_verified
2026-02-04T12:34:28.149Z
68060fe82da29557ede62815
https://cdn-avatars.huggingface.co/v1/production/uploads/68060fe82da29557ede62815/CZAMALCB2M3A35Fmid_eM.png
Huang233
Enoch-A
user
Enoch-A
697145b5c1c7409747bf94c8
Zixuan Zhong
claimed_verified
2026-02-24T09:52:06.771Z
694524196bedf03d6b02cf22
/avatars/733247ed969843cda789e710240477f1.svg
Zixuan ZHONG
Zixuan8642
user
Zixuan8642
697145b5c1c7409747bf94c9
Zhongwei Wan
697145b5c1c7409747bf94ca
Donghao Zhou
claimed_verified
2026-01-22T08:45:36.605Z
67136093d2e50f1e8c9fad52
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/0q49MyGuav8lJ9CIeyLhu.png
Donghao Zhou
donghao-zhou
user
donghao-zhou
697145b5c1c7409747bf94cb
Samiul Alam
697145b5c1c7409747bf94cc
Xin Wang
697145b5c1c7409747bf94cd
Zexin Li
697145b5c1c7409747bf94ce
Zhihao Dou
697145b5c1c7409747bf94cf
Li Zhu
697145b5c1c7409747bf94d0
Jing Xiong
697145b5c1c7409747bf94d1
Chaofan Tao
697145b5c1c7409747bf94d2
Yan Xu
697145b5c1c7409747bf94d3
Dimitrios Dimitriadis
697145b5c1c7409747bf94d4
Tuo Zhang
697145b5c1c7409747bf94d5
Mi Zhang
MMDeepResearch-Bench: A Benchmark for Multimodal Deep Research Agents
Deep Research Agents (DRAs) generate citation-rich reports via multi-step search and synthesis, yet existing benchmarks mainly target text-only settings or short-form multimodal QA, missing end-to-end multimodal evidence use. We introduce MMDeepResearch-Bench (MMDR-Bench), a benchmark of 140 expert-crafted tasks across 21 domains, where each task provides an image-text bundle to evaluate multimodal understanding and citation-grounded report generation. Compared to prior setups, MMDR-Bench emphasizes report-style synthesis with explicit evidence use, where models must connect visual artifacts to sourced claims and maintain consistency across narrative, citations, and visual references. We further propose a unified, interpretable evaluation pipeline: Formula-LLM Adaptive Evaluation (FLAE) for report quality, Trustworthy Retrieval-Aligned Citation Evaluation (TRACE) for citation-grounded evidence alignment, and Multimodal Support-Aligned Integrity Check (MOSAIC) for text-visual integrity, each producing fine-grained signals that support error diagnosis beyond a single overall score. Experiments across 25 state-of-the-art models reveal systematic trade-offs between generation quality, citation discipline, and multimodal grounding, highlighting that strong prose alone does not guarantee faithful evidence use and that multimodal integrity remains a key bottleneck for deep research agents.
MMDeepResearch-Bench evaluates multimodal research agents on report generation with visual evidence, revealing trade-offs between prose quality, citation accuracy, and visual grounding.
multimodal evidence use citation-grounded report generation multimodal understanding deep research agents Formula-LLM Adaptive Evaluation Trustworthy Retrieval-Aligned Citation Evaluation Multimodal Support-Aligned Integrity Check