PaperSearchQA: Learning to Search and Reason over Scientific Papers with RLVR
Search agents are language models (LMs) that reason and search knowledge bases (or the web) to answer questions; recent methods supervise only the final answer accuracy using reinforcement learning with verifiable rewards (RLVR). Most RLVR search agents tackle general-domain QA, which limits their relevance to technical AI systems in science, engineering, and medicine. In this work we propose training agents to search and reason over scientific papers -- this tests technical question-answering, it is directly relevant to real scientists, and the capabilities will be crucial to future AI Scientist systems. Concretely, we release a search corpus of 16 million biomedical paper abstracts and construct a challenging factoid QA dataset called PaperSearchQA with 60k samples answerable from the corpus, along with benchmarks. We train search agents in this environment to outperform non-RL retrieval baselines; we also perform further quantitative analysis and observe interesting agent behaviors like planning, reasoning, and self-verification. Our corpus, datasets, and benchmarks are usable with the popular Search-R1 codebase for RLVR training and released on https://huggingface.co/collections/jmhb/papersearchqa. Finally, our data creation methods are scalable and easily extendable to other scientific domains.
6979279adf44b75fa47e46bc
James Burgess
claimed_verified
2026-02-04T12:34:09.480Z
650871aeb44445e9b3625c7b
https://cdn-avatars.huggingface.co/v1/production/uploads/650871aeb44445e9b3625c7b/mtx3EnkuNF4z29IosnhaQ.png
James Burgess
jmhb
user
jmhb
6979279adf44b75fa47e46bd
Jan N. Hansen
6979279adf44b75fa47e46be
Duo Peng
claimed_verified
2026-02-05T10:57:17.400Z
6682599523b532718e4bd56e
/avatars/89eeaa25b10edffc33a53ed1f07a0ed2.svg
Duo Peng
pengxunduo
user
pengxunduo
6979279adf44b75fa47e46bf
Yuhui Zhang
claimed_verified
2026-02-04T12:34:07.233Z
62da55164398e21bf7f0e292
https://cdn-avatars.huggingface.co/v1/production/uploads/62da55164398e21bf7f0e292/xjKkG8IA2IZZqCdjApSh3.jpeg
Yuhui Zhang
yuhuizhang
user
yuhuizhang
6979279adf44b75fa47e46c0
Alejandro Lozano
6979279adf44b75fa47e46c1
Min Woo Sun
6979279adf44b75fa47e46c2
Emma Lundberg
6979279adf44b75fa47e46c3
Serena Yeung-Levy
PaperSearchQA: Learning to Search and Reason over Scientific Papers with RLVR
Search agents are language models (LMs) that reason and search knowledge bases (or the web) to answer questions; recent methods supervise only the final answer accuracy using reinforcement learning with verifiable rewards (RLVR). Most RLVR search agents tackle general-domain QA, which limits their relevance to technical AI systems in science, engineering, and medicine. In this work we propose training agents to search and reason over scientific papers -- this tests technical question-answering, it is directly relevant to real scientists, and the capabilities will be crucial to future AI Scientist systems. Concretely, we release a search corpus of 16 million biomedical paper abstracts and construct a challenging factoid QA dataset called PaperSearchQA with 60k samples answerable from the corpus, along with benchmarks. We train search agents in this environment to outperform non-RL retrieval baselines; we also perform further quantitative analysis and observe interesting agent behaviors like planning, reasoning, and self-verification. Our corpus, datasets, and benchmarks are usable with the popular Search-R1 codebase for RLVR training and released on https://huggingface.co/collections/jmhb/papersearchqa. Finally, our data creation methods are scalable and easily extendable to other scientific domains.
Search agents trained on scientific paper corpora demonstrate advanced reasoning capabilities for technical question-answering tasks, outperforming traditional retrieval methods through reinforcement learning with verifiable rewards.
language models reinforcement learning with verifiable rewards search agents knowledge bases scientific papers biomedical paper abstracts factoid QA PaperSearchQA RLVR Search-R1