Learning Query-Specific Rubrics from Human Preferences for DeepResearch Report Generation
Nowadays, training and evaluating DeepResearch-generated reports remain challenging due to the lack of verifiable reward signals. Accordingly, rubric-based evaluation has become a common practice. However, existing approaches either rely on coarse, pre-defined rubrics that lack sufficient granularity, or depend on manually constructed query-specific rubrics that are costly and difficult to scale. In this paper, we propose a pipeline to train human-preference-aligned query-specific rubric generators tailored for DeepResearch report generation. We first construct a dataset of DeepResearch-style queries annotated with human preferences over paired reports, and train rubric generators via reinforcement learning with a hybrid reward combining human preference supervision and LLM-based rubric evaluation. To better handle long-horizon reasoning, we further introduce a Multi-agent Markov-state (MaMs) workflow for report generation. We empirically show that our proposed rubric generators deliver more discriminative and better human-aligned supervision than existing rubric design strategies. Moreover, when integrated into the MaMs training framework, DeepResearch systems equipped with our rubric generators consistently outperform all open-source baselines on the DeepResearch Bench and achieve performance comparable to that of leading closed-source models.
6982b7fc9084cb4f0ecb5670
Changze Lv
claimed_verified
2026-02-05T10:55:23.698Z
60efa4da8432bc401cd0abc6
/avatars/3c8d8db9aaa5bd5fd1f870ac0a6b655a.svg
Changze Lv
fdu-lcz
user
fdu-lcz
6982b7fc9084cb4f0ecb5671
Jie Zhou
6982b7fc9084cb4f0ecb5672
Wentao Zhao
6982b7fc9084cb4f0ecb5673
Jingwen Xu
6982b7fc9084cb4f0ecb5674
Zisu Huang
claimed_verified
2026-02-09T14:31:45.999Z
662b6a8f0b7f23f3c000559e
/avatars/0a5b4e09ac9a8e40342131319ff32b29.svg
Zisu Huang
zisuh
user
zisuh
6982b7fc9084cb4f0ecb5675
Muzhao Tian
6982b7fc9084cb4f0ecb5676
Shihan Dou
6982b7fc9084cb4f0ecb5677
Tao Gui
6982b7fc9084cb4f0ecb5678
Le Tian
6982b7fc9084cb4f0ecb5679
Xiao Zhou
6982b7fc9084cb4f0ecb567a
Xiaoqing Zheng
6982b7fc9084cb4f0ecb567b
Xuanjing Huang
6982b7fc9084cb4f0ecb567c
Jie Zhou
Learning Query-Specific Rubrics from Human Preferences for DeepResearch Report Generation
Nowadays, training and evaluating DeepResearch-generated reports remain challenging due to the lack of verifiable reward signals. Accordingly, rubric-based evaluation has become a common practice. However, existing approaches either rely on coarse, pre-defined rubrics that lack sufficient granularity, or depend on manually constructed query-specific rubrics that are costly and difficult to scale. In this paper, we propose a pipeline to train human-preference-aligned query-specific rubric generators tailored for DeepResearch report generation. We first construct a dataset of DeepResearch-style queries annotated with human preferences over paired reports, and train rubric generators via reinforcement learning with a hybrid reward combining human preference supervision and LLM-based rubric evaluation. To better handle long-horizon reasoning, we further introduce a Multi-agent Markov-state (MaMs) workflow for report generation. We empirically show that our proposed rubric generators deliver more discriminative and better human-aligned supervision than existing rubric design strategies. Moreover, when integrated into the MaMs training framework, DeepResearch systems equipped with our rubric generators consistently outperform all open-source baselines on the DeepResearch Bench and achieve performance comparable to that of leading closed-source models.
DeepResearch report generation is improved through human-preference-aligned rubric generators trained via reinforcement learning with hybrid rewards and enhanced with multi-agent Markov-state workflows.
reinforcement learning hybrid reward human preference supervision LLM-based rubric evaluation Multi-agent Markov-state DeepResearch bench open-source baselines closed-source models