When Actions Go Off-Task: Detecting and Correcting Misaligned Actions in Computer-Use Agents
Computer-use agents (CUAs) have made tremendous progress in the past year, yet they still frequently produce misaligned actions that deviate from the user's original intent. Such misaligned actions may arise from external attacks (e.g., indirect prompt injection) or from internal limitations (e.g., erroneous reasoning). They not only expose CUAs to safety risks, but also degrade task efficiency and reliability. This work makes the first effort to define and study misaligned action detection in CUAs, with comprehensive coverage of both externally induced and internally arising misaligned actions. We further identify three common categories in real-world CUA deployment and construct MisActBench, a benchmark of realistic trajectories with human-annotated, action-level alignment labels. Moreover, we propose DeAction, a practical and universal guardrail that detects misaligned actions before execution and iteratively corrects them through structured feedback. DeAction outperforms all existing baselines across offline and online evaluations with moderate latency overhead: (1) On MisActBench, it outperforms baselines by over 15% absolute in F1 score; (2) In online evaluation, it reduces attack success rate by over 90% under adversarial settings while preserving or even improving task success rate in benign environments.
698ab3181b2dc6b37d61af84
Yuting Ning
claimed_verified
2026-02-12T13:57:56.664Z
65ace92f64c9b93eca5c2bce
https://cdn-avatars.huggingface.co/v1/production/uploads/65ace92f64c9b93eca5c2bce/pG0JRXH-8zEy0IoaEnMNw.jpeg
Yuting Ning
nnnyt
user
nnnyt
698ab3181b2dc6b37d61af85
Jaylen Jones
claimed_verified
2026-02-10T09:06:25.356Z
637029f831af06da86518bc4
/avatars/b569b77e7f261ef5dc0b072fed61a5ba.svg
Jaylen Jones 
jjones62202
user
jjones62202
698ab3181b2dc6b37d61af86
Zhehao Zhang
698ab3181b2dc6b37d61af87
Chentao Ye
698ab3181b2dc6b37d61af88
Weitong Ruan
698ab3181b2dc6b37d61af89
Junyi Li
698ab3181b2dc6b37d61af8a
Rahul Gupta
698ab3181b2dc6b37d61af8b
Huan Sun
When Actions Go Off-Task: Detecting and Correcting Misaligned Actions in Computer-Use Agents
Computer-use agents (CUAs) have made tremendous progress in the past year, yet they still frequently produce misaligned actions that deviate from the user's original intent. Such misaligned actions may arise from external attacks (e.g., indirect prompt injection) or from internal limitations (e.g., erroneous reasoning). They not only expose CUAs to safety risks, but also degrade task efficiency and reliability. This work makes the first effort to define and study misaligned action detection in CUAs, with comprehensive coverage of both externally induced and internally arising misaligned actions. We further identify three common categories in real-world CUA deployment and construct MisActBench, a benchmark of realistic trajectories with human-annotated, action-level alignment labels. Moreover, we propose DeAction, a practical and universal guardrail that detects misaligned actions before execution and iteratively corrects them through structured feedback. DeAction outperforms all existing baselines across offline and online evaluations with moderate latency overhead: (1) On MisActBench, it outperforms baselines by over 15% absolute in F1 score; (2) In online evaluation, it reduces attack success rate by over 90% under adversarial settings while preserving or even improving task success rate in benign environments.
Computer-use agents face safety risks from misaligned actions caused by external attacks or internal limitations, prompting the development of DeAction, a guardrail that detects and corrects such actions before execution.
computer-use agents misaligned actions prompt injection erroneous reasoning guardrail structured feedback attack success rate task success rate