Generative Visual Code Mobile World Models
Mobile Graphical User Interface (GUI) World Models (WMs) offer a promising path for improving mobile GUI agent performance at train- and inference-time. However, current approaches face a critical trade-off: text-based WMs sacrifice visual fidelity, while the inability of visual WMs in precise text rendering led to their reliance on slow, complex pipelines dependent on numerous external models. We propose a novel paradigm: visual world modeling via renderable code generation, where a single Vision-Language Model (VLM) predicts the next GUI state as executable web code that renders to pixels, rather than generating pixels directly. This combines the strengths of both approaches: VLMs retain their linguistic priors for precise text rendering while their pre-training on structured web code enables high-fidelity visual generation. We introduce gWorld (8B, 32B), the first open-weight visual mobile GUI WMs built on this paradigm, along with a data generation framework (gWorld) that automatically synthesizes code-based training data. In extensive evaluation across 4 in- and 2 out-of-distribution benchmarks, gWorld sets a new pareto frontier in accuracy versus model size, outperforming 8 frontier open-weight models over 50.25x larger. Further analyses show that (1) scaling training data via gWorld yields meaningful gains, (2) each component of our pipeline improves data quality, and (3) stronger world modeling improves downstream mobile GUI policy performance.
69818253ce18b186280962e1
Woosung Koh
claimed_verified
2026-02-04T12:32:16.711Z
63f5f7754b831cc179ba5410
https://cdn-avatars.huggingface.co/v1/production/uploads/1677064273670-63f5f7754b831cc179ba5410.jpeg
Reiss Koh
Reiss
user
Reiss
69818253ce18b186280962e2
Sungjun Han
claimed_verified
2026-02-04T12:32:19.770Z
67aaee60a8192c1ba3d7d42b
https://cdn-avatars.huggingface.co/v1/production/uploads/67aaee60a8192c1ba3d7d42b/2pBbWeqUEUyB82EPJpNX4.jpeg
Sungjun Han
sungjunhan-trl
user
sungjunhan-trl
69818253ce18b186280962e3
Segyu Lee
claimed_verified
2026-02-04T12:32:13.928Z
67d823d86d6baf84fba04392
/avatars/8dfe7800aa3f9d9166963a1f5da5657a.svg
segyu lee
segyulee
user
segyulee
69818253ce18b186280962e4
Se-Young Yun
69818253ce18b186280962e5
Jamin Shin
Generative Visual Code Mobile World Models
Mobile Graphical User Interface (GUI) World Models (WMs) offer a promising path for improving mobile GUI agent performance at train- and inference-time. However, current approaches face a critical trade-off: text-based WMs sacrifice visual fidelity, while the inability of visual WMs in precise text rendering led to their reliance on slow, complex pipelines dependent on numerous external models. We propose a novel paradigm: visual world modeling via renderable code generation, where a single Vision-Language Model (VLM) predicts the next GUI state as executable web code that renders to pixels, rather than generating pixels directly. This combines the strengths of both approaches: VLMs retain their linguistic priors for precise text rendering while their pre-training on structured web code enables high-fidelity visual generation. We introduce gWorld (8B, 32B), the first open-weight visual mobile GUI WMs built on this paradigm, along with a data generation framework (gWorld) that automatically synthesizes code-based training data. In extensive evaluation across 4 in- and 2 out-of-distribution benchmarks, gWorld sets a new pareto frontier in accuracy versus model size, outperforming 8 frontier open-weight models over 50.25x larger. Further analyses show that (1) scaling training data via gWorld yields meaningful gains, (2) each component of our pipeline improves data quality, and (3) stronger world modeling improves downstream mobile GUI policy performance.
Visual world models for mobile GUI agents are improved through renderable code generation using vision-language models, achieving better performance with reduced model size compared to existing approaches.
world models vision-language models renderable code generation mobile GUI agents visual fidelity text rendering code-based training data gWorld pareto frontier downstream policy performance