SAGE: Steerable Agentic Data Generation for Deep Search with Execution Feedback
Deep search agents, which aim to answer complex questions requiring reasoning across multiple documents, can significantly speed up the information-seeking process. Collecting human annotations for this application is prohibitively expensive due to long and complex exploration trajectories. We propose an agentic pipeline that automatically generates high quality, difficulty-controlled deep search question-answer pairs for a given corpus and a target difficulty level. Our pipeline, SAGE, consists of a data generator which proposes QA pairs and a search agent which attempts to solve the generated question and provide execution feedback for the data generator. The two components interact over multiple rounds to iteratively refine the question-answer pairs until they satisfy the target difficulty level. Our intrinsic evaluation shows SAGE generates questions that require diverse reasoning strategies, while significantly increases the correctness and difficulty of the generated data. Our extrinsic evaluation demonstrates up to 23% relative performance gain on popular deep search benchmarks by training deep search agents with our synthetic data. Additional experiments show that agents trained on our data can adapt from fixed-corpus retrieval to Google Search at inference time, without further training.
69783a29026bdf0473116f00
Fangyuan Xu
admin_assigned
2026-01-27T14:32:13.461Z
619d59d765db93fe9d894d97
/avatars/d9ce5e3ab62acdb669d0d9aaf328588e.svg
Fangyuan Xu
fangyuan
user
fangyuan
69783a29026bdf0473116f01
Rujun Han
admin_assigned
2026-01-27T14:32:23.311Z
64c04feeb746fe51543c1b7a
/avatars/77b87fca59a64c2463d652d454d78b13.svg
Han
Rujun
user
Rujun
69783a29026bdf0473116f02
Yanfei Chen
admin_assigned
2026-01-27T14:32:30.277Z
64dbe2aa978d1f9fafe978e7
/avatars/5e046834aa1f3ec88fa0fb3e5e3f0825.svg
Yanfei Chen
anmourchen
user
anmourchen
69783a29026bdf0473116f03
Zifeng Wang
69783a29026bdf0473116f04
I-Hung Hsu
admin_assigned
2026-01-27T14:32:39.995Z
64dc08f9e7bc8544f9b1ac32
/avatars/8d6c321318753c49a82ee485ef7b74ea.svg
I-Hung Hsu
ihungalexhsu
user
ihungalexhsu
69783a29026bdf0473116f05
Jun Yan
69783a29026bdf0473116f06
Vishy Tirumalashetty
69783a29026bdf0473116f07
Eunsol Choi
admin_assigned
2026-01-27T14:32:51.393Z
5fca68d111d483aef7d3b26c
/avatars/f563d906f9caf52838df7d7fb31f7c90.svg
Eunsol Choi
eunsol
user
eunsol
69783a29026bdf0473116f08
Tomas Pfister
admin_assigned
2026-01-27T14:33:01.573Z
68e3f17160026cedfe38266b
/avatars/ebecb599788f53d98076d4e37a060f19.svg
Tomas Pfister
tpfister
user
tpfister
69783a29026bdf0473116f09
Chen-Yu Lee
SAGE: Steerable Agentic Data Generation for Deep Search with Execution Feedback
Deep search agents, which aim to answer complex questions requiring reasoning across multiple documents, can significantly speed up the information-seeking process. Collecting human annotations for this application is prohibitively expensive due to long and complex exploration trajectories. We propose an agentic pipeline that automatically generates high quality, difficulty-controlled deep search question-answer pairs for a given corpus and a target difficulty level. Our pipeline, SAGE, consists of a data generator which proposes QA pairs and a search agent which attempts to solve the generated question and provide execution feedback for the data generator. The two components interact over multiple rounds to iteratively refine the question-answer pairs until they satisfy the target difficulty level. Our intrinsic evaluation shows SAGE generates questions that require diverse reasoning strategies, while significantly increases the correctness and difficulty of the generated data. Our extrinsic evaluation demonstrates up to 23% relative performance gain on popular deep search benchmarks by training deep search agents with our synthetic data. Additional experiments show that agents trained on our data can adapt from fixed-corpus retrieval to Google Search at inference time, without further training.
Deep search agents trained on synthetic question-answer pairs generated through an iterative agent-based pipeline demonstrate improved performance and adaptability across different search environments.
deep search agents question-answer pairs reasoning strategies intrinsic evaluation extrinsic evaluation synthetic data agent-based pipeline iterative refinement