Step-GUI Technical Report
Recent advances in multimodal large language models unlock unprecedented opportunities for GUI automation. However, a fundamental challenge remains: how to efficiently acquire high-quality training data while maintaining annotation reliability? We introduce a self-evolving training pipeline powered by the Calibrated Step Reward System, which converts model-generated trajectories into reliable training signals through trajectory-level calibration, achieving >90% annotation accuracy with 10-100x lower cost. Leveraging this pipeline, we introduce Step-GUI, a family of models (4B/8B) that achieves state-of-the-art GUI performance (8B: 80.2% AndroidWorld, 48.5% OSWorld, 62.6% ScreenShot-Pro) while maintaining robust general capabilities. As GUI agent capabilities improve, practical deployment demands standardized interfaces across heterogeneous devices while protecting user privacy. To this end, we propose GUI-MCP, the first Model Context Protocol for GUI automation with hierarchical architecture that combines low-level atomic operations and high-level task delegation to local specialist models, enabling high-privacy execution where sensitive data stays on-device. Finally, to assess whether agents can handle authentic everyday usage, we introduce AndroidDaily, a benchmark grounded in real-world mobile usage patterns with 3146 static actions and 235 end-to-end tasks across high-frequency daily scenarios (8B: static 89.91%, end-to-end 52.50%). Our work advances the development of practical GUI agents and demonstrates strong potential for real-world deployment in everyday digital interactions.
69437417542d62d58a7bf6c4
Haolong Yan
69437417542d62d58a7bf6c5
Jia Wang
69437417542d62d58a7bf6c6
Xin Huang
69437417542d62d58a7bf6c7
Yeqing Shen
69437417542d62d58a7bf6c8
Ziyang Meng
claimed_verified
2025-12-18T07:59:53.033Z
653614073f4248157d60ccdc
/avatars/c9298bab1cdc1d0b6ffe4c7c5ef18bd5.svg
mengziyang
zylate
user
zylate
69437417542d62d58a7bf6c9
Zhimin Fan
69437417542d62d58a7bf6ca
Kaijun Tan
69437417542d62d58a7bf6cb
Jin Gao
69437417542d62d58a7bf6cc
Lieyu Shi
69437417542d62d58a7bf6cd
Mi Yang
69437417542d62d58a7bf6ce
Shiliang Yang
69437417542d62d58a7bf6cf
Zhirui Wang
69437417542d62d58a7bf6d0
Brian Li
69437417542d62d58a7bf6d1
Kang An
69437417542d62d58a7bf6d2
Chenyang Li
69437417542d62d58a7bf6d3
Lei Lei
69437417542d62d58a7bf6d4
Mengmeng Duan
69437417542d62d58a7bf6d5
Danxun Liang
69437417542d62d58a7bf6d6
Guodong Liu
69437417542d62d58a7bf6d7
Hang Cheng
69437417542d62d58a7bf6d8
Hao Wu
69437417542d62d58a7bf6d9
Jie Dong
69437417542d62d58a7bf6da
Junhao Huang
69437417542d62d58a7bf6db
Mei Chen
69437417542d62d58a7bf6dc
Renjie Yu
69437417542d62d58a7bf6dd
Shunshan Li
69437417542d62d58a7bf6de
Xu Zhou
69437417542d62d58a7bf6df
Yiting Dai
69437417542d62d58a7bf6e0
Yineng Deng
69437417542d62d58a7bf6e1
Yingdan Liang
69437417542d62d58a7bf6e2
Zelin Chen
69437417542d62d58a7bf6e3
Wen Sun
69437417542d62d58a7bf6e4
Chengxu Yan
69437417542d62d58a7bf6e5
Chunqin Xu
69437417542d62d58a7bf6e6
Dong Li
69437417542d62d58a7bf6e7
Fengqiong Xiao
69437417542d62d58a7bf6e8
Guanghao Fan
69437417542d62d58a7bf6e9
Guopeng Li
69437417542d62d58a7bf6ea
Guozhen Peng
69437417542d62d58a7bf6eb
Hongbing Li
69437417542d62d58a7bf6ec
Hang Li
69437417542d62d58a7bf6ed
Hongming Chen
69437417542d62d58a7bf6ee
Jingjing Xie
69437417542d62d58a7bf6ef
Jianyong Li
69437417542d62d58a7bf6f0
Jingyang Zhang
69437417542d62d58a7bf6f1
Jiaju Ren
69437417542d62d58a7bf6f2
Jiayu Yuan
69437417542d62d58a7bf6f3
Jianpeng Yin
69437417542d62d58a7bf6f4
Kai Cao
69437417542d62d58a7bf6f5
Liang Zhao
69437417542d62d58a7bf6f6
Liguo Tan
69437417542d62d58a7bf6f7
Liying Shi
69437417542d62d58a7bf6f8
Mengqiang Ren
69437417542d62d58a7bf6f9
Min Xu
69437417542d62d58a7bf6fa
Manjiao Liu
69437417542d62d58a7bf6fb
Mao Luo
69437417542d62d58a7bf6fc
Mingxin Wan
69437417542d62d58a7bf6fd
Na Wang
69437417542d62d58a7bf6fe
Nan Wu
69437417542d62d58a7bf6ff
Ning Wang
69437417542d62d58a7bf700
Peiyao Ma
69437417542d62d58a7bf701
Qingzhou Zhang
69437417542d62d58a7bf702
Qiao Wang
69437417542d62d58a7bf703
Qinlin Zeng
69437417542d62d58a7bf704
Qiong Gao
69437417542d62d58a7bf705
Qiongyao Li
69437417542d62d58a7bf706
Shangwu Zhong
69437417542d62d58a7bf707
Shuli Gao
69437417542d62d58a7bf708
Shaofan Liu
69437417542d62d58a7bf709
Shisi Gao
69437417542d62d58a7bf70a
Shuang Luo
69437417542d62d58a7bf70b
Xingbin Liu
69437417542d62d58a7bf70c
Xiaojia Liu
69437417542d62d58a7bf70d
Xiaojie Hou
69437417542d62d58a7bf70e
Xin Liu
69437417542d62d58a7bf70f
Xuanti Feng
69437417542d62d58a7bf710
Xuedan Cai
69437417542d62d58a7bf711
Xuan Wen
69437417542d62d58a7bf712
Xianwei Zhu
69437417542d62d58a7bf713
Xin Liang
69437417542d62d58a7bf714
Xin Liu
69437417542d62d58a7bf715
Xin Zhou
69437417542d62d58a7bf716
Yingxiu Zhao
69437417542d62d58a7bf717
Yukang Shi
69437417542d62d58a7bf718
Yunfang Xu
69437417542d62d58a7bf719
Yuqing Zeng
69437417542d62d58a7bf71a
Yixun Zhang
69437417542d62d58a7bf71b
Zejia Weng
69437417542d62d58a7bf71c
Zhonghao Yan
claimed_verified
2026-01-08T08:33:38.773Z
64fbd4e69a62bb2791b3a665
https://cdn-avatars.huggingface.co/v1/production/uploads/64fbd4e69a62bb2791b3a665/ZEMtU8O0z98ryeRCG3l_K.jpeg
Zhonghao Yan
zzzyzh
user
zzzyzh
69437417542d62d58a7bf71d
Zhiguo Huang
69437417542d62d58a7bf71e
Zhuoyu Wang
69437417542d62d58a7bf71f
Zheng Ge
69437417542d62d58a7bf720
Jing Li
69437417542d62d58a7bf721
Yibo Zhu
69437417542d62d58a7bf722
Binxing Jiao
69437417542d62d58a7bf723
Xiangyu Zhang
69437417542d62d58a7bf724
Daxin Jiang
Step-GUI Technical Report
Recent advances in multimodal large language models unlock unprecedented opportunities for GUI automation. However, a fundamental challenge remains: how to efficiently acquire high-quality training data while maintaining annotation reliability? We introduce a self-evolving training pipeline powered by the Calibrated Step Reward System, which converts model-generated trajectories into reliable training signals through trajectory-level calibration, achieving >90% annotation accuracy with 10-100x lower cost. Leveraging this pipeline, we introduce Step-GUI, a family of models (4B/8B) that achieves state-of-the-art GUI performance (8B: 80.2% AndroidWorld, 48.5% OSWorld, 62.6% ScreenShot-Pro) while maintaining robust general capabilities. As GUI agent capabilities improve, practical deployment demands standardized interfaces across heterogeneous devices while protecting user privacy. To this end, we propose GUI-MCP, the first Model Context Protocol for GUI automation with hierarchical architecture that combines low-level atomic operations and high-level task delegation to local specialist models, enabling high-privacy execution where sensitive data stays on-device. Finally, to assess whether agents can handle authentic everyday usage, we introduce AndroidDaily, a benchmark grounded in real-world mobile usage patterns with 3146 static actions and 235 end-to-end tasks across high-frequency daily scenarios (8B: static 89.91%, end-to-end 52.50%). Our work advances the development of practical GUI agents and demonstrates strong potential for real-world deployment in everyday digital interactions.
A self-evolving training pipeline with the Calibrated Step Reward System and GUI-MCP protocol improve GUI automation efficiency, accuracy, and privacy in real-world scenarios.
multimodal large language models GUI automation self-evolving training pipeline Calibrated Step Reward System trajectory-level calibration Step-GUI GUI performance GUI-MCP Model Context Protocol AndroidWorld OSWorld ScreenShot-Pro AndroidDaily real-world mobile usage patterns hierarchical architecture low-level atomic operations high-level task delegation local specialist models high-privacy execution