Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learning
Recent advances in large language model (LLM) have empowered autonomous agents to perform complex tasks that require multi-turn interactions with tools and environments. However, scaling such agent training is limited by the lack of diverse and reliable environments. In this paper, we propose Agent World Model (AWM), a fully synthetic environment generation pipeline. Using this pipeline, we scale to 1,000 environments covering everyday scenarios, in which agents can interact with rich toolsets (35 tools per environment on average) and obtain high-quality observations. Notably, these environments are code-driven and backed by databases, providing more reliable and consistent state transitions than environments simulated by LLMs. Moreover, they enable more efficient agent interaction compared with collecting trajectories from realistic environments. To demonstrate the effectiveness of this resource, we perform large-scale reinforcement learning for multi-turn tool-use agents. Thanks to the fully executable environments and accessible database states, we can also design reliable reward functions. Experiments on three benchmarks show that training exclusively in synthetic environments, rather than benchmark-specific ones, yields strong out-of-distribution generalization. The code is available at https://github.com/Snowflake-Labs/agent-world-model.
698bf6fe6052d3bed9630ac0
Zhaoyang Wang
698bf6fe6052d3bed9630ac1
Canwen Xu
698bf6fe6052d3bed9630ac2
Boyi Liu
698bf6fe6052d3bed9630ac3
Yite Wang
698bf6fe6052d3bed9630ac4
Siwei Han
698bf6fe6052d3bed9630ac5
Zhewei Yao
698bf6fe6052d3bed9630ac6
Huaxiu Yao
claimed_verified
2026-03-19T13:04:59.254Z
65f667a455009c4ad9e6ac4c
/avatars/ed10c2cf2ba3fde6d7da93f076961607.svg
Yao
Huaxiu
user
Huaxiu
698bf6fe6052d3bed9630ac7
Yuxiong He
Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learning
Recent advances in large language model (LLM) have empowered autonomous agents to perform complex tasks that require multi-turn interactions with tools and environments. However, scaling such agent training is limited by the lack of diverse and reliable environments. In this paper, we propose Agent World Model (AWM), a fully synthetic environment generation pipeline. Using this pipeline, we scale to 1,000 environments covering everyday scenarios, in which agents can interact with rich toolsets (35 tools per environment on average) and obtain high-quality observations. Notably, these environments are code-driven and backed by databases, providing more reliable and consistent state transitions than environments simulated by LLMs. Moreover, they enable more efficient agent interaction compared with collecting trajectories from realistic environments. To demonstrate the effectiveness of this resource, we perform large-scale reinforcement learning for multi-turn tool-use agents. Thanks to the fully executable environments and accessible database states, we can also design reliable reward functions. Experiments on three benchmarks show that training exclusively in synthetic environments, rather than benchmark-specific ones, yields strong out-of-distribution generalization. The code is available at https://github.com/Snowflake-Labs/agent-world-model.
Large language model agents trained in synthetic environments with code-driven simulations and database-backed state transitions demonstrate superior out-of-distribution generalization compared to traditional benchmark-specific approaches.
large language model autonomous agents multi-turn interactions tool-use agents reinforcement learning synthetic environment generation code-driven environments database-backed state transitions out-of-distribution generalization