World Models for Policy Refinement in StarCraft II
Large Language Models (LLMs) have recently shown strong reasoning and generalization capabilities, motivating their use as decision-making policies in complex environments. StarCraft II (SC2), with its massive state-action space and partial observability, is a challenging testbed. However, existing LLM-based SC2 agents primarily focus on improving the policy itself and overlook integrating a learnable, action-conditioned transition model into the decision loop. To bridge this gap, we propose StarWM, the first world model for SC2 that predicts future observations under partial observability. To facilitate learning SC2's hybrid dynamics, we introduce a structured textual representation that factorizes observations into five semantic modules, and construct SC2-Dynamics-50k, the first instruction-tuning dataset for SC2 dynamics prediction. We further develop a multi-dimensional offline evaluation framework for predicted structured observations. Offline results show StarWM's substantial gains over zero-shot baselines, including nearly 60% improvements in resource prediction accuracy and self-side macro-situation consistency. Finally, we propose StarWM-Agent, a world-model-augmented decision system that integrates StarWM into a Generate--Simulate--Refine decision loop for foresight-driven policy refinement. Online evaluation against SC2's built-in AI demonstrates consistent improvements, yielding win-rate gains of 30%, 15%, and 30% against Hard (LV5), Harder (LV6), and VeryHard (LV7), respectively, alongside improved macro-management stability and tactical risk assessment.
6997f8677a658569d5a10242
Yixin Zhang
claimed_verified
2026-02-20T08:32:38.896Z
6588fe10cef9b8827b1d0c82
https://cdn-avatars.huggingface.co/v1/production/uploads/6588fe10cef9b8827b1d0c82/8vr9HnlvGa20p48k4oIVZ.jpeg
Yixin Zhang
yxzhang2024
user
yxzhang2024
6997f8677a658569d5a10243
Ziyi Wang
6997f8677a658569d5a10244
Yiming Rong
6997f8677a658569d5a10245
Haoxi Wang
6997f8677a658569d5a10246
Jinling Jiang
6997f8677a658569d5a10247
Shuang Xu
6997f8677a658569d5a10248
Haoran Wu
6997f8677a658569d5a10249
Shiyu Zhou
6997f8677a658569d5a1024a
Bo Xu
World Models for Policy Refinement in StarCraft II
Large Language Models (LLMs) have recently shown strong reasoning and generalization capabilities, motivating their use as decision-making policies in complex environments. StarCraft II (SC2), with its massive state-action space and partial observability, is a challenging testbed. However, existing LLM-based SC2 agents primarily focus on improving the policy itself and overlook integrating a learnable, action-conditioned transition model into the decision loop. To bridge this gap, we propose StarWM, the first world model for SC2 that predicts future observations under partial observability. To facilitate learning SC2's hybrid dynamics, we introduce a structured textual representation that factorizes observations into five semantic modules, and construct SC2-Dynamics-50k, the first instruction-tuning dataset for SC2 dynamics prediction. We further develop a multi-dimensional offline evaluation framework for predicted structured observations. Offline results show StarWM's substantial gains over zero-shot baselines, including nearly 60% improvements in resource prediction accuracy and self-side macro-situation consistency. Finally, we propose StarWM-Agent, a world-model-augmented decision system that integrates StarWM into a Generate--Simulate--Refine decision loop for foresight-driven policy refinement. Online evaluation against SC2's built-in AI demonstrates consistent improvements, yielding win-rate gains of 30%, 15%, and 30% against Hard (LV5), Harder (LV6), and VeryHard (LV7), respectively, alongside improved macro-management stability and tactical risk assessment.
StarWM, a world model for StarCraft II, predicts future observations under partial observability using a structured textual representation and achieves significant performance improvements in win rate and decision-making stability.
world model StarCraft II partial observability action-conditioned transition model structured textual representation SC2-Dynamics-50k offline evaluation Generate-Simulate-Refine loop foresight-driven policy refinement