From Word to World: Can Large Language Models be Implicit Text-based World Models?
Agentic reinforcement learning increasingly relies on experience-driven scaling, yet real-world environments remain non-adaptive, limited in coverage, and difficult to scale. World models offer a potential way to improve learning efficiency through simulated experience, but it remains unclear whether large language models can reliably serve this role and under what conditions they meaningfully benefit agents. We study these questions in text-based environments, which provide a controlled setting to reinterpret language modeling as next-state prediction under interaction. We introduce a three-level framework for evaluating LLM-based world models: (i) fidelity and consistency, (ii) scalability and robustness, and (iii) agent utility. Across five representative environments, we find that sufficiently trained world models maintain coherent latent state, scale predictably with data and model size, and improve agent performance via action verification, synthetic trajectory generation, and warm-starting reinforcement learning. Meanwhile, these gains depend critically on behavioral coverage and environment complexity, delineating clear boundry on when world modeling effectively supports agent learning.
6949fc14335742716e9321d5
Yixia Li
claimed_verified
2025-12-25T20:50:07.465Z
6645bdf6621ded608be9c37e
https://cdn-avatars.huggingface.co/v1/production/uploads/6645bdf6621ded608be9c37e/lHFUPBkqCyKc6KCaBUaCs.jpeg
Yixia Li
X1AOX1A
user
X1AOX1A
6949fc14335742716e9321d6
Hongru Wang
claimed_verified
2025-12-25T20:50:04.919Z
65f906e5c3dbdcae83ff7aac
https://cdn-avatars.huggingface.co/v1/production/uploads/65f906e5c3dbdcae83ff7aac/mdjiVkLDJgJcGLwv0rMe4.jpeg
Hongru Wang
Merlin-Hongru
user
Merlin-Hongru
6949fc14335742716e9321d7
Jiahao Qiu
6949fc14335742716e9321d8
Zhenfei Yin
6949fc14335742716e9321d9
Dongdong Zhang
6949fc14335742716e9321da
Cheng Qian
6949fc14335742716e9321db
Zeping Li
6949fc14335742716e9321dc
Pony Ma
6949fc14335742716e9321dd
Guanhua Chen
6949fc14335742716e9321de
Heng Ji
6949fc14335742716e9321df
Mengdi Wang
From Word to World: Can Large Language Models be Implicit Text-based World Models?
Agentic reinforcement learning increasingly relies on experience-driven scaling, yet real-world environments remain non-adaptive, limited in coverage, and difficult to scale. World models offer a potential way to improve learning efficiency through simulated experience, but it remains unclear whether large language models can reliably serve this role and under what conditions they meaningfully benefit agents. We study these questions in text-based environments, which provide a controlled setting to reinterpret language modeling as next-state prediction under interaction. We introduce a three-level framework for evaluating LLM-based world models: (i) fidelity and consistency, (ii) scalability and robustness, and (iii) agent utility. Across five representative environments, we find that sufficiently trained world models maintain coherent latent state, scale predictably with data and model size, and improve agent performance via action verification, synthetic trajectory generation, and warm-starting reinforcement learning. Meanwhile, these gains depend critically on behavioral coverage and environment complexity, delineating clear boundry on when world modeling effectively supports agent learning.
LLM-based world models enhance agent performance in text-based environments through action verification, synthetic trajectory generation, and warm-starting reinforcement learning, but their effectiveness is contingent on behavioral coverage and environment complexity.
agentic reinforcement learning experience-driven scaling real-world environments world models large language models next-state prediction three-level framework fidelity consistency scalability robustness agent utility action verification synthetic trajectory generation warm-starting reinforcement learning