UI-Voyager: A Self-Evolving GUI Agent Learning via Failed Experience
Autonomous mobile GUI agents have attracted increasing attention along with the advancement of Multimodal Large Language Models (MLLMs). However, existing methods still suffer from inefficient learning from failed trajectories and ambiguous credit assignment under sparse rewards for long-horizon GUI tasks. To that end, we propose UI-Voyager, a novel two-stage self-evolving mobile GUI agent. In the first stage, we employ Rejection Fine-Tuning (RFT), which enables the continuous co-evolution of data and models in a fully autonomous loop. The second stage introduces Group Relative Self-Distillation (GRSD), which identifies critical fork points in group rollouts and constructs dense step-level supervision from successful trajectories to correct failed ones. Extensive experiments on AndroidWorld show that our 4B model achieves an 81.0% Pass@1 success rate, outperforming numerous recent baselines and exceeding human-level performance. Ablation and case studies further verify the effectiveness of GRSD. Our method represents a significant leap toward efficient, self-evolving, and high-performance mobile GUI automation without expensive manual data annotation.
69c49c004ae7eef815b9d9be
Zichuan Lin
claimed_verified
2026-03-26T13:42:37.054Z
687da36e2eaea8261f1323d6
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/hFH69bJGIDMikEYyClray.png
zichuan lin
zichuan-lin
user
zichuan-lin
69c49c004ae7eef815b9d9bf
Feiyu Liu
claimed_verified
2026-04-01T08:36:20.760Z
67adefc85d76edd3b22e3383
/avatars/f1068718b085743ac75c68c515f81eca.svg
Liu
MarsXL
user
MarsXL
69c49c004ae7eef815b9d9c0
Yijun Yang
claimed_verified
2026-03-26T13:42:39.143Z
645e553c3b6d85c65e8b0e54
/avatars/1fffc6499b9d65b21a895ca96f03b781.svg
Steven
yijunyang
user
yijunyang
69c49c004ae7eef815b9d9c1
Jiafei Lyu
69c49c004ae7eef815b9d9c2
Yiming Gao
claimed_verified
2026-03-26T13:42:43.429Z
67cff2a43a9d50150f6d55e6
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/iY05tc7uuocF55aT08Vdx.png
Yatming Gor
code-yatming
user
code-yatming
69c49c004ae7eef815b9d9c3
Yicheng Liu
claimed_verified
2026-03-26T13:42:41.756Z
68be47de9dd8d73e702c376b
/avatars/3bc26a294eecc63372ee77abb821fed7.svg
yichengliu
yiceason
user
yiceason
69c49c004ae7eef815b9d9c4
Zhicong Lu
69c49c004ae7eef815b9d9c5
Yangbin Yu
69c49c004ae7eef815b9d9c6
Mingyu Yang
69c49c004ae7eef815b9d9c7
Junyou Li
69c49c004ae7eef815b9d9c8
Deheng Ye
69c49c004ae7eef815b9d9c9
Jie Jiang
UI-Voyager: A Self-Evolving GUI Agent Learning via Failed Experience
Autonomous mobile GUI agents have attracted increasing attention along with the advancement of Multimodal Large Language Models (MLLMs). However, existing methods still suffer from inefficient learning from failed trajectories and ambiguous credit assignment under sparse rewards for long-horizon GUI tasks. To that end, we propose UI-Voyager, a novel two-stage self-evolving mobile GUI agent. In the first stage, we employ Rejection Fine-Tuning (RFT), which enables the continuous co-evolution of data and models in a fully autonomous loop. The second stage introduces Group Relative Self-Distillation (GRSD), which identifies critical fork points in group rollouts and constructs dense step-level supervision from successful trajectories to correct failed ones. Extensive experiments on AndroidWorld show that our 4B model achieves an 81.0% Pass@1 success rate, outperforming numerous recent baselines and exceeding human-level performance. Ablation and case studies further verify the effectiveness of GRSD. Our method represents a significant leap toward efficient, self-evolving, and high-performance mobile GUI automation without expensive manual data annotation.
A two-stage self-evolving mobile GUI agent named UI-Voyager is proposed, featuring rejection fine-tuning and group relative self-distillation to improve efficiency and performance in GUI automation tasks.
Rejection Fine-Tuning Group Relative Self-Distillation mobile GUI agent multimodal large language models GUI tasks autonomous loop dense step-level supervision AndroidWorld