Code2World: A GUI World Model via Renderable Code Generation
Autonomous GUI agents interact with environments by perceiving interfaces and executing actions. As a virtual sandbox, the GUI World model empowers agents with human-like foresight by enabling action-conditioned prediction. However, existing text- and pixel-based approaches struggle to simultaneously achieve high visual fidelity and fine-grained structural controllability. To this end, we propose Code2World, a vision-language coder that simulates the next visual state via renderable code generation. Specifically, to address the data scarcity problem, we construct AndroidCode by translating GUI trajectories into high-fidelity HTML and refining synthesized code through a visual-feedback revision mechanism, yielding a corpus of over 80K high-quality screen-action pairs. To adapt existing VLMs into code prediction, we first perform SFT as a cold start for format layout following, then further apply Render-Aware Reinforcement Learning which uses rendered outcome as the reward signal by enforcing visual semantic fidelity and action consistency. Extensive experiments demonstrate that Code2World-8B achieves the top-performing next UI prediction, rivaling the competitive GPT-5 and Gemini-3-Pro-Image. Notably, Code2World significantly enhances downstream navigation success rates in a flexible manner, boosting Gemini-2.5-Flash by +9.5% on AndroidWorld navigation. The code is available at https://github.com/AMAP-ML/Code2World.
698bf5b66052d3bed9630aa7
Yuhao Zheng
claimed_verified
2026-02-11T11:14:28.241Z
64107c7df52d7eb22e062956
/avatars/7b1cee9a2b8454fedfbd4c3d1df9865c.svg
Yuhao Zheng
yhzheng1031
user
yhzheng1031
698bf5b66052d3bed9630aa8
Li'an Zhong
698bf5b66052d3bed9630aa9
Yi Wang
claimed_verified
2026-02-11T11:14:30.778Z
6773bcaa675a971ddf1e81dd
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/a8VUwZYXd7O_mq_zFvXMh.png
CokeWang
CokeWang
user
CokeWang
698bf5b66052d3bed9630aaa
Rui Dai
claimed_verified
2026-02-11T11:14:25.982Z
661de9defdbc9c247f159d15
/avatars/38e21e78327cc908201122405c48f41b.svg
Rui Dai
DerryD
user
DerryD
698bf5b66052d3bed9630aab
Kaikui Liu
698bf5b66052d3bed9630aac
Xiangxiang Chu
698bf5b66052d3bed9630aad
Linyuan Lv
698bf5b66052d3bed9630aae
Philip Torr
698bf5b66052d3bed9630aaf
Kevin Qinghong Lin
claimed_verified
2026-02-11T11:14:23.397Z
64440be5af034cdfd69ca3a7
https://cdn-avatars.huggingface.co/v1/production/uploads/64440be5af034cdfd69ca3a7/qmx24QiDFT29vleCxL9TX.jpeg
Qinghong (Kevin) Lin
KevinQHLin
user
KevinQHLin
Code2World: A GUI World Model via Renderable Code Generation
Autonomous GUI agents interact with environments by perceiving interfaces and executing actions. As a virtual sandbox, the GUI World model empowers agents with human-like foresight by enabling action-conditioned prediction. However, existing text- and pixel-based approaches struggle to simultaneously achieve high visual fidelity and fine-grained structural controllability. To this end, we propose Code2World, a vision-language coder that simulates the next visual state via renderable code generation. Specifically, to address the data scarcity problem, we construct AndroidCode by translating GUI trajectories into high-fidelity HTML and refining synthesized code through a visual-feedback revision mechanism, yielding a corpus of over 80K high-quality screen-action pairs. To adapt existing VLMs into code prediction, we first perform SFT as a cold start for format layout following, then further apply Render-Aware Reinforcement Learning which uses rendered outcome as the reward signal by enforcing visual semantic fidelity and action consistency. Extensive experiments demonstrate that Code2World-8B achieves the top-performing next UI prediction, rivaling the competitive GPT-5 and Gemini-3-Pro-Image. Notably, Code2World significantly enhances downstream navigation success rates in a flexible manner, boosting Gemini-2.5-Flash by +9.5% on AndroidWorld navigation. The code is available at https://github.com/AMAP-ML/Code2World.
Code2World enables autonomous GUI agents to predict next visual states through renderable code generation, achieving high visual fidelity and structural controllability while improving navigation performance.
vision-language coder GUI World model action-conditioned prediction AndroidCode HTML generation visual-feedback revision mechanism SFT Render-Aware Reinforcement Learning visual semantic fidelity action consistency next UI prediction AndroidWorld navigation