Retrieval-Infused Reasoning Sandbox: A Benchmark for Decoupling Retrieval and Reasoning Capabilities
Despite strong performance on existing benchmarks, it remains unclear whether large language models can reason over genuinely novel scientific information. Most evaluations score end-to-end RAG pipelines, where reasoning is confounded with retrieval and toolchain choices, and the signal is further contaminated by parametric memorization and open-web volatility. We introduce DeR2, a controlled deep-research sandbox that isolates document-grounded reasoning while preserving core difficulties of deep search: multi-step synthesis, denoising, and evidence-based conclusion making. DeR2 decouples evidence access from reasoning via four regimes--Instruction-only, Concepts (gold concepts without documents), Related-only (only relevant documents), and Full-set (relevant documents plus topically related distractors)--yielding interpretable regime gaps that operationalize retrieval loss vs. reasoning loss and enable fine-grained error attribution. To prevent parametric leakage, we apply a two-phase validation that requires parametric failure without evidence while ensuring oracle-concept solvability. To ensure reproducibility, each instance provides a frozen document library (drawn from 2023-2025 theoretical papers) with expert-annotated concepts and validated rationales. Experiments across a diverse set of state-of-the-art foundation models reveal substantial variation and significant headroom: some models exhibit mode-switch fragility, performing worse with the Full-set than with Instruction-only, while others show structural concept misuse, correctly naming concepts but failing to execute them as procedures.
6985651d4ad556f294b7ebd9
Shuangshuang Ying
6985651d4ad556f294b7ebda
Zheyu Wang
claimed_verified
2026-02-09T08:35:34.349Z
68a6c5f3c48e21c896cfb701
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/UexEznOwvwivI-k7Xe2Qa.png
Zheyu Wang
FilianoreZY
user
FilianoreZY
6985651d4ad556f294b7ebdb
Yunjian Peng
6985651d4ad556f294b7ebdc
Jin Chen
6985651d4ad556f294b7ebdd
Yuhao Wu
6985651d4ad556f294b7ebde
Hongbin Lin
6985651d4ad556f294b7ebdf
Dingyu He
6985651d4ad556f294b7ebe0
Siyi Liu
6985651d4ad556f294b7ebe1
Gengchen Yu
6985651d4ad556f294b7ebe2
YinZhu Piao
6985651d4ad556f294b7ebe3
Yuchen Wu
6985651d4ad556f294b7ebe4
Xin Gui
claimed_verified
2026-02-06T18:51:40.436Z
67dbf07f9d821d38905d145d
/avatars/e806467d2c0f4f642c8d4906b0855817.svg
guixin
Ross12
user
Ross12
6985651d4ad556f294b7ebe5
Zhongyuan Peng
6985651d4ad556f294b7ebe6
Xin Li
6985651d4ad556f294b7ebe7
Xeron Du
6985651d4ad556f294b7ebe8
Libo Qin
6985651d4ad556f294b7ebe9
YiXin Cao
6985651d4ad556f294b7ebea
Ge Zhang
claimed_verified
2026-02-06T18:51:37.889Z
638efcf4c67af472d316d424
/avatars/97a57859d7d87a3a8f1bb41d32a72bc2.svg
Ge Zhang
zhangysk
user
zhangysk
6985651d4ad556f294b7ebeb
Stephen Huang
Retrieval-Infused Reasoning Sandbox: A Benchmark for Decoupling Retrieval and Reasoning Capabilities
Despite strong performance on existing benchmarks, it remains unclear whether large language models can reason over genuinely novel scientific information. Most evaluations score end-to-end RAG pipelines, where reasoning is confounded with retrieval and toolchain choices, and the signal is further contaminated by parametric memorization and open-web volatility. We introduce DeR2, a controlled deep-research sandbox that isolates document-grounded reasoning while preserving core difficulties of deep search: multi-step synthesis, denoising, and evidence-based conclusion making. DeR2 decouples evidence access from reasoning via four regimes--Instruction-only, Concepts (gold concepts without documents), Related-only (only relevant documents), and Full-set (relevant documents plus topically related distractors)--yielding interpretable regime gaps that operationalize retrieval loss vs. reasoning loss and enable fine-grained error attribution. To prevent parametric leakage, we apply a two-phase validation that requires parametric failure without evidence while ensuring oracle-concept solvability. To ensure reproducibility, each instance provides a frozen document library (drawn from 2023-2025 theoretical papers) with expert-annotated concepts and validated rationales. Experiments across a diverse set of state-of-the-art foundation models reveal substantial variation and significant headroom: some models exhibit mode-switch fragility, performing worse with the Full-set than with Instruction-only, while others show structural concept misuse, correctly naming concepts but failing to execute them as procedures.
DeR2 presents a controlled evaluation framework for assessing language models' document-grounded reasoning capabilities by isolating reasoning from retrieval and toolchain decisions.
large language models document-grounded reasoning retrieval-augmented generation deep search multi-step synthesis denoising evidence-based conclusion making parametric memorization open-web volatility controlled deep-research sandbox retrieval loss reasoning loss error attribution two-phase validation parametric failure oracle-concept solvability frozen document library expert-annotated concepts validated rationales