Tool-R0: Self-Evolving LLM Agents for Tool-Learning from Zero Data
Large language models (LLMs) are becoming the foundation for autonomous agents that can use tools to solve complex tasks. Reinforcement learning (RL) has emerged as a common approach for injecting such agentic capabilities, but typically under tightly controlled training setups. It often depends on carefully constructed task-solution pairs and substantial human supervision, which creates a fundamental obstacle to open-ended self-evolution toward superintelligent systems. In this paper, we propose Tool-R0 framework for training general purpose tool-calling agents from scratch with self-play RL, under a zero-data assumption. Initialized from the same base LLM, Tool-R0 co-evolves a Generator and a Solver with complementary rewards: one proposes targeted challenging tasks at the other's competence frontier and the other learns to solve them with real-world tool calls. This creates a self-evolving cycle that requires no pre-existing tasks or datasets. Evaluation on different tool-use benchmarks show that Tool-R0 yields 92.5 relative improvement over the base model and surpasses fully supervised tool-calling baselines under the same setting. Our work further provides empirical insights into self-play LLM agents by analyzing co-evolution, curriculum dynamics, and scaling behavior.
69a004a6ebfce7fbcca91c85
Emre Can Acikgoz
69a004a6ebfce7fbcca91c86
Cheng Qian
69a004a6ebfce7fbcca91c87
Jonas Hübotter
claimed_verified
2026-03-04T09:24:49.538Z
6645dbab80ee4eb5ddc8aed8
https://cdn-avatars.huggingface.co/v1/production/uploads/6645dbab80ee4eb5ddc8aed8/-IckaooWMXeACKew9THv_.jpeg
Jonas Hübotter
jonhue
user
jonhue
69a004a6ebfce7fbcca91c88
Heng Ji
69a004a6ebfce7fbcca91c89
Dilek Hakkani-Tür
69a004a6ebfce7fbcca91c8a
Gokhan Tur
Tool-R0: Self-Evolving LLM Agents for Tool-Learning from Zero Data
Large language models (LLMs) are becoming the foundation for autonomous agents that can use tools to solve complex tasks. Reinforcement learning (RL) has emerged as a common approach for injecting such agentic capabilities, but typically under tightly controlled training setups. It often depends on carefully constructed task-solution pairs and substantial human supervision, which creates a fundamental obstacle to open-ended self-evolution toward superintelligent systems. In this paper, we propose Tool-R0 framework for training general purpose tool-calling agents from scratch with self-play RL, under a zero-data assumption. Initialized from the same base LLM, Tool-R0 co-evolves a Generator and a Solver with complementary rewards: one proposes targeted challenging tasks at the other's competence frontier and the other learns to solve them with real-world tool calls. This creates a self-evolving cycle that requires no pre-existing tasks or datasets. Evaluation on different tool-use benchmarks show that Tool-R0 yields 92.5 relative improvement over the base model and surpasses fully supervised tool-calling baselines under the same setting. Our work further provides empirical insights into self-play LLM agents by analyzing co-evolution, curriculum dynamics, and scaling behavior.
Tool-R0 framework enables training general-purpose tool-calling agents through self-play reinforcement learning without initial datasets, achieving significant performance improvements over base models and supervised baselines.
large language models reinforcement learning self-play tool-calling agents zero-data assumption co-evolution curriculum dynamics scaling behavior