DREAM: Deep Research Evaluation with Agentic Metrics
Deep Research Agents generate analyst-grade reports, yet evaluating them remains challenging due to the absence of a single ground truth and the multidimensional nature of research quality. Recent benchmarks propose distinct methodologies, yet they suffer from the Mirage of Synthesis, where strong surface-level fluency and citation alignment can obscure underlying factual and reasoning defects. We characterize this gap by introducing a taxonomy across four verticals that exposes a critical capability mismatch: static evaluators inherently lack the tool-use capabilities required to assess temporal validity and factual correctness. To address this, we propose DREAM (Deep Research Evaluation with Agentic Metrics), a framework that instantiates the principle of capability parity by making evaluation itself agentic. DREAM structures assessment through an evaluation protocol combining query-agnostic metrics with adaptive metrics generated by a tool-calling agent, enabling temporally aware coverage, grounded verification, and systematic reasoning probes. Controlled evaluations demonstrate DREAM is significantly more sensitive to factual and temporal decay than existing benchmarks, offering a scalable, reference-free evaluation paradigm.
699e9a47dfbcf0b800aecbd8
Elad Ben Avraham
699e9a47dfbcf0b800aecbd9
Changhao Li
claimed_verified
2026-02-27T16:43:32.346Z
660026d9573abbcdb975aa13
https://cdn-avatars.huggingface.co/v1/production/uploads/660026d9573abbcdb975aa13/XktfFxTMC_d1xJ4pDJLum.jpeg
Changhao
lichangh20
user
lichangh20
699e9a47dfbcf0b800aecbda
Ron Dorfman
699e9a47dfbcf0b800aecbdb
Roy Ganz
699e9a47dfbcf0b800aecbdc
Oren Nuriel
699e9a47dfbcf0b800aecbdd
Amir Dudai
699e9a47dfbcf0b800aecbde
Aviad Aberdam
699e9a47dfbcf0b800aecbdf
Noah Flynn
699e9a47dfbcf0b800aecbe0
Elman Mansimov
699e9a47dfbcf0b800aecbe1
Adi Kalyanpur
699e9a47dfbcf0b800aecbe2
Ron Litman
DREAM: Deep Research Evaluation with Agentic Metrics
Deep Research Agents generate analyst-grade reports, yet evaluating them remains challenging due to the absence of a single ground truth and the multidimensional nature of research quality. Recent benchmarks propose distinct methodologies, yet they suffer from the Mirage of Synthesis, where strong surface-level fluency and citation alignment can obscure underlying factual and reasoning defects. We characterize this gap by introducing a taxonomy across four verticals that exposes a critical capability mismatch: static evaluators inherently lack the tool-use capabilities required to assess temporal validity and factual correctness. To address this, we propose DREAM (Deep Research Evaluation with Agentic Metrics), a framework that instantiates the principle of capability parity by making evaluation itself agentic. DREAM structures assessment through an evaluation protocol combining query-agnostic metrics with adaptive metrics generated by a tool-calling agent, enabling temporally aware coverage, grounded verification, and systematic reasoning probes. Controlled evaluations demonstrate DREAM is significantly more sensitive to factual and temporal decay than existing benchmarks, offering a scalable, reference-free evaluation paradigm.
Deep Research Agents generate analyst-grade reports, yet evaluating them remains challenging due to the absence of a single ground truth and the multidimensional nature of research quality. Recent benchmarks propose distinct methodologies, yet they suffer from the Mirage of Synthesis, where strong surface-level fluency and citation alignment can obscure underlying factual and reasoning defects. We characterize this gap by introducing a taxonomy across four verticals that exposes a critical capability mismatch: static evaluators inherently lack the tool-use capabilities required to assess temporal validity and factual correctness. To address this, we propose DREAM (Deep Research Evaluation with Agentic Metrics), a framework that instantiates the principle of capability parity by making evaluation itself agentic. DREAM structures assessment through an evaluation protocol combining query-agnostic metrics with adaptive metrics generated by a tool-calling agent, enabling temporally aware coverage, grounded verification, and systematic reasoning probes. Controlled evaluations demonstrate DREAM is significantly more sensitive to factual and temporal decay than existing benchmarks, offering a scalable, reference-free evaluation paradigm.
Deep Research Agents analyst-grade reports Mirage of Synthesis tool-use capabilities temporal validity factual correctness DREAM agentic metrics evaluation protocol query-agnostic metrics adaptive metrics tool-calling agent temporally aware coverage grounded verification systematic reasoning probes