SimWorld: An Open-ended Realistic Simulator for Autonomous Agents in Physical and Social Worlds
While LLM/VLM-powered AI agents have advanced rapidly in math, coding, and computer use, their applications in complex physical and social environments remain challenging. Building agents that can survive and thrive in the real world (for example, by autonomously earning income or running a business) requires massive-scale interaction, reasoning, training, and evaluation across diverse embodied scenarios. However, existing world simulators for such development fall short: they often rely on limited hand-crafted environments, simulate simplified game-like physics and social rules, and lack native support for LLM/VLM agents. We introduce SimWorld, a new simulator built on Unreal Engine 5, designed for developing and evaluating LLM/VLM agents in rich, real-world-like settings. SimWorld offers three core capabilities: (1) realistic, open-ended world simulation, including accurate physical and social dynamics and language-driven procedural environment generation; (2) a rich interface for LLM/VLM agents, with multimodal world inputs and open-vocabulary actions at varying levels of abstraction; and (3) diverse and extensible physical and social reasoning scenarios that are easily customizable by users. We demonstrate SimWorld by deploying frontier LLM agents (e.g., GPT-4o, Gemini-2.5-Flash, Claude-3.5, and DeepSeek-Prover-V2) on long-horizon multi-agent delivery tasks involving strategic cooperation and competition. The results reveal distinct reasoning patterns and limitations across models. We open-source SimWorld and hope it becomes a foundational platform for advancing real-world agent intelligence across disciplines: https://simworld.org.
692ffba226742347f61db184
Jiawei Ren
692ffba226742347f61db185
Yan Zhuang
692ffba226742347f61db186
Xiaokang Ye
claimed_verified
2025-12-04T08:48:24.594Z
6700678116bb14dfc9750d02
https://cdn-avatars.huggingface.co/v1/production/uploads/6700678116bb14dfc9750d02/zPVHXcMsD_fPfK0CbdtX2.jpeg
XiaokangYe
KoeYe
user
KoeYe
692ffba226742347f61db187
Lingjun Mao
claimed_verified
2025-12-04T08:48:27.174Z
65509836e5f380aca800c408
https://cdn-avatars.huggingface.co/v1/production/uploads/65509836e5f380aca800c408/wS6edLhx3YNaWWj0o8kaV.jpeg
Lingjun Mao
mao1207
user
mao1207
692ffba226742347f61db188
Xuhong He
692ffba226742347f61db189
Jianzhi Shen
692ffba226742347f61db18a
Mrinaal Dogra
claimed_verified
2025-12-04T08:50:29.235Z
67310ef1bbc14f43a4e9c283
/avatars/9ed1e6b8daa848e7f5dd3a111ef3ac69.svg
Mrinaal Dogra
mdogra
user
mdogra
692ffba226742347f61db18b
Yiming Liang
claimed_verified
2025-12-08T08:33:00.488Z
654fed4d7490049d621d0cd3
/avatars/6bc1553d1571ab4e9ab80c58eeb9820c.svg
Yiming Liang
Yiming1234
user
Yiming1234
692ffba226742347f61db18c
Ruixuan Zhang
692ffba226742347f61db18d
Tianai Yue
692ffba226742347f61db18e
Yiqing Yang
692ffba226742347f61db18f
Eric Liu
692ffba226742347f61db190
Ryan Wu
692ffba226742347f61db191
Kevin Benavente
692ffba226742347f61db192
Rajiv Mandya Nagaraju
692ffba226742347f61db193
Muhammad Faayez
692ffba226742347f61db194
Xiyan Zhang
692ffba226742347f61db195
Dhruv Vivek Sharma
692ffba226742347f61db196
Xianrui Zhong
692ffba226742347f61db197
Ziqiao Ma
claimed_verified
2025-12-22T11:01:18.055Z
630cfc45b66f088d547b2768
https://cdn-avatars.huggingface.co/v1/production/uploads/630cfc45b66f088d547b2768/9-dMts2xFVbPmPHJGGqBx.png
Martin Ziqiao Ma
marstin
user
marstin
692ffba226742347f61db198
Tianmin Shu
692ffba226742347f61db199
Zhiting Hu
692ffba226742347f61db19a
Lianhui Qin
SimWorld: An Open-ended Realistic Simulator for Autonomous Agents in Physical and Social Worlds
While LLM/VLM-powered AI agents have advanced rapidly in math, coding, and computer use, their applications in complex physical and social environments remain challenging. Building agents that can survive and thrive in the real world (for example, by autonomously earning income or running a business) requires massive-scale interaction, reasoning, training, and evaluation across diverse embodied scenarios. However, existing world simulators for such development fall short: they often rely on limited hand-crafted environments, simulate simplified game-like physics and social rules, and lack native support for LLM/VLM agents. We introduce SimWorld, a new simulator built on Unreal Engine 5, designed for developing and evaluating LLM/VLM agents in rich, real-world-like settings. SimWorld offers three core capabilities: (1) realistic, open-ended world simulation, including accurate physical and social dynamics and language-driven procedural environment generation; (2) a rich interface for LLM/VLM agents, with multimodal world inputs and open-vocabulary actions at varying levels of abstraction; and (3) diverse and extensible physical and social reasoning scenarios that are easily customizable by users. We demonstrate SimWorld by deploying frontier LLM agents (e.g., GPT-4o, Gemini-2.5-Flash, Claude-3.5, and DeepSeek-Prover-V2) on long-horizon multi-agent delivery tasks involving strategic cooperation and competition. The results reveal distinct reasoning patterns and limitations across models. We open-source SimWorld and hope it becomes a foundational platform for advancing real-world agent intelligence across disciplines: https://simworld.org.
SimWorld, a new Unreal Engine 5-based simulator, enables the development and evaluation of LLM/VLM agents in realistic, real-world-like settings with diverse physical and social reasoning scenarios.
LLM VLM AI agents world simulation Unreal Engine 5 procedural environment generation multimodal inputs open-vocabulary actions reasoning scenarios multi-agent delivery tasks strategic cooperation competition DeepSeek-Prover-V2 GPT-4o Gemini-2.5-Flash Claude-3.5