CaMeLs Can Use Computers Too: System-level Security for Computer Use Agents
AI agents are vulnerable to prompt injection attacks, where malicious content hijacks agent behavior to steal credentials or cause financial loss. The only known robust defense is architectural isolation that strictly separates trusted task planning from untrusted environment observations. However, applying this design to Computer Use Agents (CUAs) -- systems that automate tasks by viewing screens and executing actions -- presents a fundamental challenge: current agents require continuous observation of UI state to determine each action, conflicting with the isolation required for security. We resolve this tension by demonstrating that UI workflows, while dynamic, are structurally predictable. We introduce Single-Shot Planning for CUAs, where a trusted planner generates a complete execution graph with conditional branches before any observation of potentially malicious content, providing provable control flow integrity guarantees against arbitrary instruction injections. Although this architectural isolation successfully prevents instruction injections, we show that additional measures are needed to prevent Branch Steering attacks, which manipulate UI elements to trigger unintended valid paths within the plan. We evaluate our design on OSWorld, and retain up to 57% of the performance of frontier models while improving performance for smaller open-source models by up to 19%, demonstrating that rigorous security and utility can coexist in CUAs.
6969f752844a787c4fdea435
Hanna Foerster
admin_assigned
2026-01-16T15:45:26.738Z
666c1544de375f3114d590c0
/avatars/de6c4d3395160603b0bf68c9bca398df.svg
Hanna Foerster
aprilflower
user
aprilflower
6969f752844a787c4fdea436
Robert Mullins
admin_assigned
2026-01-16T15:45:33.009Z
66349b8dc53115681c534d07
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/Suu9bZJecfCcJccjRsrqH.png
Robert Mullins
rcmullins
user
rcmullins
6969f752844a787c4fdea437
Tom Blanchard
admin_assigned
2026-01-16T15:45:40.522Z
664a1d42640bc89c935f191e
/avatars/a7a40b51564c43ed595840945e0a58f8.svg
Tom Blanchard
tom-bl
user
tom-bl
6969f752844a787c4fdea438
Nicolas Papernot
6969f752844a787c4fdea439
Kristina Nikolić
admin_assigned
2026-01-16T15:45:50.298Z
6695455330bd2a19adca14c0
/avatars/a183124c6f13f1179f202c7462ef5bf7.svg
Kristina Nikolic
nkristina
user
nkristina
6969f752844a787c4fdea43a
Florian Tramèr
admin_assigned
2026-01-16T15:45:56.278Z
63568f18ba90b4ea9fe91cb5
/avatars/3e8b3c573e20cf80d329a312bfc34728.svg
Florian Tramer
ftramer
user
ftramer
6969f752844a787c4fdea43b
Ilia Shumailov
6969f752844a787c4fdea43c
Cheng Zhang
6969f752844a787c4fdea43d
Yiren Zhao
admin_assigned
2026-01-16T15:46:05.784Z
6654464df0c8c891ca528618
/avatars/26aa1f673da92e3632576954bcee1b28.svg
Yiren Zhao
YirenZH
user
YirenZH
CaMeLs Can Use Computers Too: System-level Security for Computer Use Agents
AI agents are vulnerable to prompt injection attacks, where malicious content hijacks agent behavior to steal credentials or cause financial loss. The only known robust defense is architectural isolation that strictly separates trusted task planning from untrusted environment observations. However, applying this design to Computer Use Agents (CUAs) -- systems that automate tasks by viewing screens and executing actions -- presents a fundamental challenge: current agents require continuous observation of UI state to determine each action, conflicting with the isolation required for security. We resolve this tension by demonstrating that UI workflows, while dynamic, are structurally predictable. We introduce Single-Shot Planning for CUAs, where a trusted planner generates a complete execution graph with conditional branches before any observation of potentially malicious content, providing provable control flow integrity guarantees against arbitrary instruction injections. Although this architectural isolation successfully prevents instruction injections, we show that additional measures are needed to prevent Branch Steering attacks, which manipulate UI elements to trigger unintended valid paths within the plan. We evaluate our design on OSWorld, and retain up to 57% of the performance of frontier models while improving performance for smaller open-source models by up to 19%, demonstrating that rigorous security and utility can coexist in CUAs.
Computer Use Agents face security challenges from prompt injection attacks, but a single-shot planning approach with architectural isolation enables secure autonomous task execution while maintaining performance.
prompt injection attacks architectural isolation Computer Use Agents UI workflows execution graph conditional branches control flow integrity instruction injections Branch Steering attacks OSWorld