DIVE: Scaling Diversity in Agentic Task Synthesis for Generalizable Tool Use
Recent work synthesizes agentic tasks for post-training tool-using LLMs, yet robust generalization under shifts in tasks and toolsets remains an open challenge. We trace this brittleness to insufficient diversity in synthesized tasks. Scaling diversity is difficult because training requires tasks to remain executable and verifiable, while generalization demands coverage of diverse tool types, toolset combinations, and heterogeneous tool-use patterns. We propose DIVE, an evidence-driven recipe that inverts synthesis order, executing diverse, real-world tools first and reverse-deriving tasks strictly entailed by the resulting traces, thereby providing grounding by construction. DIVE scales structural diversity along two controllable axes, tool-pool coverage and per-task toolset variety, and an Evidence Collection--Task Derivation loop further induces rich multi-step tool-use patterns across 373 tools in five domains. Training Qwen3-8B on DIVE data (48k SFT + 3.2k RL) improves by +22 average points across 9 OOD benchmarks and outperforms the strongest 8B baseline by +68. Remarkably, controlled scaling analysis reveals that diversity scaling consistently outperforms quantity scaling for OOD generalization, even with 4x less data.
69b376752ad6c5d4cf486a43
Aili Chen
claimed_verified
2026-03-13T08:12:53.843Z
63f86b099f87cc3e645b51d9
/avatars/27ca5ba425640bf67474cee871e8e53a.svg
Ellie Chen
sheep33333
user
sheep33333
69b376752ad6c5d4cf486a44
Chi Zhang
69b376752ad6c5d4cf486a45
Junteng Liu
69b376752ad6c5d4cf486a46
Jiangjie Chen
69b376752ad6c5d4cf486a47
Chengyu Du
69b376752ad6c5d4cf486a48
Yunji Li
69b376752ad6c5d4cf486a49
Ming Zhong
69b376752ad6c5d4cf486a4a
Qin Wang
69b376752ad6c5d4cf486a4b
Zhengmao Zhu
69b376752ad6c5d4cf486a4c
Jiayuan Song
69b376752ad6c5d4cf486a4d
Ke Ji
69b376752ad6c5d4cf486a4e
Junxian He
69b376752ad6c5d4cf486a4f
Pengyu Zhao
69b376752ad6c5d4cf486a50
Yanghua Xiao
DIVE: Scaling Diversity in Agentic Task Synthesis for Generalizable Tool Use
Recent work synthesizes agentic tasks for post-training tool-using LLMs, yet robust generalization under shifts in tasks and toolsets remains an open challenge. We trace this brittleness to insufficient diversity in synthesized tasks. Scaling diversity is difficult because training requires tasks to remain executable and verifiable, while generalization demands coverage of diverse tool types, toolset combinations, and heterogeneous tool-use patterns. We propose DIVE, an evidence-driven recipe that inverts synthesis order, executing diverse, real-world tools first and reverse-deriving tasks strictly entailed by the resulting traces, thereby providing grounding by construction. DIVE scales structural diversity along two controllable axes, tool-pool coverage and per-task toolset variety, and an Evidence Collection--Task Derivation loop further induces rich multi-step tool-use patterns across 373 tools in five domains. Training Qwen3-8B on DIVE data (48k SFT + 3.2k RL) improves by +22 average points across 9 OOD benchmarks and outperforms the strongest 8B baseline by +68. Remarkably, controlled scaling analysis reveals that diversity scaling consistently outperforms quantity scaling for OOD generalization, even with 4x less data.
Training Qwen3-8B on DIVE data improves performance across out-of-distribution benchmarks, with diversity scaling outperforming quantity scaling even with less data.
agentic tasks post-training tool-using LLMs robust generalization task synthesis evidence-driven recipe structural diversity tool-pool coverage per-task toolset variety Evidence Collection--Task Derivation loop multi-step tool-use patterns out-of-distribution generalization diversity scaling quantity scaling