DR Tulu: Reinforcement Learning with Evolving Rubrics for Deep Research
Deep research models perform multi-step research to produce long-form, well-attributed answers. However, most open deep research models are trained on easily verifiable short-form QA tasks via reinforcement learning with verifiable rewards (RLVR), which does not extend to realistic long-form tasks. We address this with Reinforcement Learning with Evolving Rubrics (RLER), in which we construct and maintain rubrics that co-evolve with the policy model during training; this allows the rubrics to incorporate information that the model has newly explored and to provide discriminative, on-policy feedback. Using RLER, we develop Deep Research Tulu (DR Tulu-8B), the first open model that is directly trained for open-ended, long-form deep research. Across four long-form deep research benchmarks in science, healthcare and general domains, DR Tulu substantially outperforms existing open deep research models, and matches or exceeds proprietary deep research systems, while being significantly smaller and cheaper per query. To facilitate future research, we release all data, models, and code, including our new MCP-based agent infrastructure for deep research systems.
692531b316eb3a9f13103a46
Rulin Shao
692531b316eb3a9f13103a47
Akari Asai
claimed_verified
2025-11-25T15:53:14.992Z
6266e0cb7a1f5a1562c4e86e
https://cdn-avatars.huggingface.co/v1/production/uploads/6266e0cb7a1f5a1562c4e86e/kkJZPssa76mdRPnKH-q3e.jpeg
Akari Asai
akariasai
user
akariasai
692531b316eb3a9f13103a48
Shannon Zejiang Shen
claimed_verified
2025-11-25T09:05:11.372Z
613f897ffbfd59f147a88c81
https://cdn-avatars.huggingface.co/v1/production/uploads/1659455519119-613f897ffbfd59f147a88c81.jpeg
Shannon Shen
shannons
user
shannons
692531b316eb3a9f13103a49
Hamish Ivison
claimed_verified
2025-11-25T09:05:13.345Z
62608fc2ffe8827cb1d89f9f
https://cdn-avatars.huggingface.co/v1/production/uploads/1654027835241-62608fc2ffe8827cb1d89f9f.png
Hamish Ivison
hamishivi
user
hamishivi
692531b316eb3a9f13103a4a
Varsha Kishore
692531b316eb3a9f13103a4b
Jingming Zhuo
claimed_verified
2025-11-25T09:34:32.721Z
6530c7263976e5f4412ba737
/avatars/8b4d9d847a9e115c3da8cad629bd0a41.svg
Jingming Zhuo
JingmingZ
user
JingmingZ
692531b316eb3a9f13103a4c
Xinran Zhao
692531b316eb3a9f13103a4d
Molly Park
692531b316eb3a9f13103a4e
Samuel G. Finlayson
692531b316eb3a9f13103a4f
David Sontag
692531b316eb3a9f13103a50
Tyler Murray
admin_assigned
2025-11-25T09:41:12.772Z
65e5fefbe8cbae176d9ca005
https://cdn-avatars.huggingface.co/v1/production/uploads/65e5fefbe8cbae176d9ca005/5SLREDsVzycEwVsPNv765.jpeg
Tyler Murray
undfined
user
undfined
692531b316eb3a9f13103a51
Sewon Min
admin_assigned
2025-11-25T09:41:07.253Z
63a76d0de27a6dbd485fe863
https://cdn-avatars.huggingface.co/v1/production/uploads/63a76d0de27a6dbd485fe863/qJJwHOuvyQGq1o0KscOF_.jpeg
Sewon Min
sewon
user
sewon
692531b316eb3a9f13103a52
Pradeep Dasigi
admin_assigned
2025-11-25T09:41:00.782Z
6408fcc93461c51cf735a61e
/avatars/619f3653911d111f046a5a6c30fc8319.svg
Pradeep Dasigi
pradeepd
user
pradeepd
692531b316eb3a9f13103a53
Luca Soldaini
claimed_verified
2026-02-04T19:04:44.793Z
5f04d8c45d08220171a0ad32
https://cdn-avatars.huggingface.co/v1/production/uploads/5f04d8c45d08220171a0ad32/B0szblazqtHQql_-pWOHW.jpeg
Luca Soldaini
soldni
user
soldni
692531b316eb3a9f13103a54
Faeze Brahman
admin_assigned
2025-11-25T09:41:17.953Z
65282b8d578679aac7888aec
https://cdn-avatars.huggingface.co/v1/production/uploads/65282b8d578679aac7888aec/dibBkhH-z1c70mJZZxJ7u.jpeg
Faeze Brahman
faezeb
user
faezeb
692531b316eb3a9f13103a55
Wen-tau Yih
692531b316eb3a9f13103a56
Tongshuang Wu
692531b316eb3a9f13103a57
Luke Zettlemoyer
692531b316eb3a9f13103a58
Yoon Kim
692531b316eb3a9f13103a59
Hannaneh Hajishirzi
692531b316eb3a9f13103a5a
Pang Wei Koh
admin_assigned
2025-11-25T09:38:49.292Z
641b4263abfce26bcf7b27de
/avatars/e91b4205e4f74b0dd8c333c23203a924.svg
Pang Wei Koh
pangwei
user
pangwei
DR Tulu: Reinforcement Learning with Evolving Rubrics for Deep Research
Deep research models perform multi-step research to produce long-form, well-attributed answers. However, most open deep research models are trained on easily verifiable short-form QA tasks via reinforcement learning with verifiable rewards (RLVR), which does not extend to realistic long-form tasks. We address this with Reinforcement Learning with Evolving Rubrics (RLER), in which we construct and maintain rubrics that co-evolve with the policy model during training; this allows the rubrics to incorporate information that the model has newly explored and to provide discriminative, on-policy feedback. Using RLER, we develop Deep Research Tulu (DR Tulu-8B), the first open model that is directly trained for open-ended, long-form deep research. Across four long-form deep research benchmarks in science, healthcare and general domains, DR Tulu substantially outperforms existing open deep research models, and matches or exceeds proprietary deep research systems, while being significantly smaller and cheaper per query. To facilitate future research, we release all data, models, and code, including our new MCP-based agent infrastructure for deep research systems.
Reinforcement Learning with Evolving Rubrics (RLER) enables training of deep research models for long-form tasks, outperforming existing models and proprietary systems while being more cost-effective.
Reinforcement Learning with Verifiable Rewards (RLVR) Reinforcement Learning with Evolving Rubrics (RLER) Deep Research Tulu (DR Tulu-8B) long-form deep research deep research models open-ended tasks science benchmarks healthcare benchmarks general domain benchmarks MCP-based agent infrastructure