OS-Sentinel: Towards Safety-Enhanced Mobile GUI Agents via Hybrid
  Validation in Realistic Workflows
Computer-using agents powered by Vision-Language Models (VLMs) have
demonstrated human-like capabilities in operating digital environments like
mobile platforms. While these agents hold great promise for advancing digital
automation, their potential for unsafe operations, such as system compromise
and privacy leakage, is raising significant concerns. Detecting these safety
concerns across the vast and complex operational space of mobile environments
presents a formidable challenge that remains critically underexplored. To
establish a foundation for mobile agent safety research, we introduce
MobileRisk-Live, a dynamic sandbox environment accompanied by a safety
detection benchmark comprising realistic trajectories with fine-grained
annotations. Built upon this, we propose OS-Sentinel, a novel hybrid safety
detection framework that synergistically combines a Formal Verifier for
detecting explicit system-level violations with a VLM-based Contextual Judge
for assessing contextual risks and agent actions. Experiments show that
OS-Sentinel achieves 10%-30% improvements over existing approaches across
multiple metrics. Further analysis provides critical insights that foster the
development of safer and more reliable autonomous mobile agents.
6901c572646208eac0d1f58b
Qiushi Sun
claimed_verified
2025-10-30T14:41:31.108Z
6064a0eeb1703ddba0d458b9
https://cdn-avatars.huggingface.co/v1/production/uploads/1617207525789-noauth.png
Qiushi
QiushiSun
user
QiushiSun
6901c572646208eac0d1f58c
Mukai Li
claimed_verified
2025-11-03T20:57:43.415Z
624561c1939c9acec2103534
/avatars/3efdf797c53d153cea7415213fb5afc7.svg
Mukai Li
kiaia
user
kiaia
6901c572646208eac0d1f58d
Zhoumianze Liu
6901c572646208eac0d1f58e
Zhihui Xie
6901c572646208eac0d1f58f
Fangzhi Xu
claimed_verified
2026-02-06T18:58:20.059Z
64e6cf78ecce34cb442dc889
https://cdn-avatars.huggingface.co/v1/production/uploads/64e6cf78ecce34cb442dc889/qVZFiUEpBpSkmH8SQeinm.jpeg
Fangzhi Xu
xufangzhi
user
xufangzhi
6901c572646208eac0d1f590
Zhangyue Yin
6901c572646208eac0d1f591
Kanzhi Cheng
claimed_verified
2025-11-03T20:57:41.222Z
63340dbbd92c5842ae71d1e9
/avatars/3a3182996bd41b526dcbfa8687d91963.svg
Kanzhi Cheng
cckevinn
user
cckevinn
6901c572646208eac0d1f592
Zehao Li
6901c572646208eac0d1f593
Zichen Ding
claimed_verified
2025-11-03T20:57:38.971Z
642b9861bb77f8456634b048
https://cdn-avatars.huggingface.co/v1/production/uploads/642b9861bb77f8456634b048/VrNmmcdgX7FufQmdP5YaG.jpeg
Zichen Ding
heroding77
user
heroding77
6901c572646208eac0d1f594
Qi Liu
6901c572646208eac0d1f595
Zhiyong Wu
6901c572646208eac0d1f596
Zhuosheng Zhang
6901c572646208eac0d1f597
Ben Kao
6901c572646208eac0d1f598
Lingpeng Kong
OS-Sentinel: Towards Safety-Enhanced Mobile GUI Agents via Hybrid
  Validation in Realistic Workflows
Computer-using agents powered by Vision-Language Models (VLMs) have
demonstrated human-like capabilities in operating digital environments like
mobile platforms. While these agents hold great promise for advancing digital
automation, their potential for unsafe operations, such as system compromise
and privacy leakage, is raising significant concerns. Detecting these safety
concerns across the vast and complex operational space of mobile environments
presents a formidable challenge that remains critically underexplored. To
establish a foundation for mobile agent safety research, we introduce
MobileRisk-Live, a dynamic sandbox environment accompanied by a safety
detection benchmark comprising realistic trajectories with fine-grained
annotations. Built upon this, we propose OS-Sentinel, a novel hybrid safety
detection framework that synergistically combines a Formal Verifier for
detecting explicit system-level violations with a VLM-based Contextual Judge
for assessing contextual risks and agent actions. Experiments show that
OS-Sentinel achieves 10%-30% improvements over existing approaches across
multiple metrics. Further analysis provides critical insights that foster the
development of safer and more reliable autonomous mobile agents.
A hybrid safety detection framework combining formal verification and VLM-based contextual assessment improves the detection of unsafe operations in mobile agents.
Vision-Language Models VLMs Formal Verifier Contextual Judge MobileRisk-Live safety detection benchmark system-level violations contextual risks autonomous mobile agents