SAGE: Benchmarking and Improving Retrieval for Deep Research Agents
Deep research agents have emerged as powerful systems for addressing complex queries. Meanwhile, LLM-based retrievers have demonstrated strong capability in following instructions or reasoning. This raises a critical question: can LLM-based retrievers effectively contribute to deep research agent workflows? To investigate this, we introduce SAGE, a benchmark for scientific literature retrieval comprising 1,200 queries across four scientific domains, with a 200,000 paper retrieval corpus.We evaluate six deep research agents and find that all systems struggle with reasoning-intensive retrieval. Using DR Tulu as backbone, we further compare BM25 and LLM-based retrievers (i.e., ReasonIR and gte-Qwen2-7B-instruct) as alternative search tools. Surprisingly, BM25 significantly outperforms LLM-based retrievers by approximately 30%, as existing agents generate keyword-oriented sub-queries. To improve performance, we propose a corpus-level test-time scaling framework that uses LLMs to augment documents with metadata and keywords, making retrieval easier for off-the-shelf retrievers. This yields 8% and 2% gains on short-form and open-ended questions, respectively.
6985737d4ad556f294b7ec60
Tiansheng Hu
claimed_verified
2026-02-06T18:50:53.683Z
67492b9e347c3876f22b3684
/avatars/0d80d23f7b10ce8bac689f6e8317a014.svg
Tiansheng Hu
HughieHu
user
HughieHu
6985737d4ad556f294b7ec61
Yilun Zhao
claimed_verified
2026-02-06T18:57:46.462Z
62f662bcc58915315c4eccea
https://cdn-avatars.huggingface.co/v1/production/uploads/62f662bcc58915315c4eccea/zOAQLONfMP88zr70sxHK-.jpeg
Yilun Zhao
yilunzhao
user
yilunzhao
6985737d4ad556f294b7ec62
Canyu Zhang
6985737d4ad556f294b7ec63
Arman Cohan
6985737d4ad556f294b7ec64
Chen Zhao
SAGE: Benchmarking and Improving Retrieval for Deep Research Agents
Deep research agents have emerged as powerful systems for addressing complex queries. Meanwhile, LLM-based retrievers have demonstrated strong capability in following instructions or reasoning. This raises a critical question: can LLM-based retrievers effectively contribute to deep research agent workflows? To investigate this, we introduce SAGE, a benchmark for scientific literature retrieval comprising 1,200 queries across four scientific domains, with a 200,000 paper retrieval corpus.We evaluate six deep research agents and find that all systems struggle with reasoning-intensive retrieval. Using DR Tulu as backbone, we further compare BM25 and LLM-based retrievers (i.e., ReasonIR and gte-Qwen2-7B-instruct) as alternative search tools. Surprisingly, BM25 significantly outperforms LLM-based retrievers by approximately 30%, as existing agents generate keyword-oriented sub-queries. To improve performance, we propose a corpus-level test-time scaling framework that uses LLMs to augment documents with metadata and keywords, making retrieval easier for off-the-shelf retrievers. This yields 8% and 2% gains on short-form and open-ended questions, respectively.
LLM-based retrievers show limited effectiveness in deep research agent workflows, with traditional BM25 performing better, though corpus-level test-time scaling can improve retrieval performance.
LLM-based retrievers deep research agents scientific literature retrieval SAGE benchmark ReasonIR gte-Qwen2-7B-instruct BM25 test-time scaling document augmentation