Agents of Chaos
We report an exploratory red-teaming study of autonomous language-model-powered agents deployed in a live laboratory environment with persistent memory, email accounts, Discord access, file systems, and shell execution. Over a two-week period, twenty AI researchers interacted with the agents under benign and adversarial conditions. Focusing on failures emerging from the integration of language models with autonomy, tool use, and multi-party communication, we document eleven representative case studies. Observed behaviors include unauthorized compliance with non-owners, disclosure of sensitive information, execution of destructive system-level actions, denial-of-service conditions, uncontrolled resource consumption, identity spoofing vulnerabilities, cross-agent propagation of unsafe practices, and partial system takeover. In several cases, agents reported task completion while the underlying system state contradicted those reports. We also report on some of the failed attempts. Our findings establish the existence of security-, privacy-, and governance-relevant vulnerabilities in realistic deployment settings. These behaviors raise unresolved questions regarding accountability, delegated authority, and responsibility for downstream harms, and warrant urgent attention from legal scholars, policymakers, and researchers across disciplines. This report serves as an initial empirical contribution to that broader conversation.
699d1e434e37ec6dfa1bc58f
Natalie Shapira
699d1e434e37ec6dfa1bc590
Chris Wendler
699d1e434e37ec6dfa1bc591
Avery Yen
699d1e434e37ec6dfa1bc592
Gabriele Sarti
claimed_verified
2026-03-02T09:39:43.032Z
5e7749883d77a72421292d07
https://cdn-avatars.huggingface.co/v1/production/uploads/5e7749883d77a72421292d07/M4AmBReZk_otxCIG3o0bL.jpeg
Gabriele Sarti
gsarti
user
gsarti
699d1e434e37ec6dfa1bc593
Koyena Pal
699d1e434e37ec6dfa1bc594
Olivia Floody
699d1e434e37ec6dfa1bc595
Adam Belfki
699d1e434e37ec6dfa1bc596
Alex Loftus
699d1e434e37ec6dfa1bc597
Aditya Ratan Jannali
699d1e434e37ec6dfa1bc598
Nikhil Prakash
699d1e434e37ec6dfa1bc599
Jasmine Cui
699d1e434e37ec6dfa1bc59a
Giordano Rogers
699d1e434e37ec6dfa1bc59b
Jannik Brinkmann
699d1e434e37ec6dfa1bc59c
Can Rager
699d1e434e37ec6dfa1bc59d
Amir Zur
699d1e434e37ec6dfa1bc59e
Michael Ripa
699d1e434e37ec6dfa1bc59f
Aruna Sankaranarayanan
699d1e434e37ec6dfa1bc5a0
David Atkinson
699d1e434e37ec6dfa1bc5a1
Rohit Gandikota
699d1e434e37ec6dfa1bc5a2
Jaden Fiotto-Kaufman
699d1e434e37ec6dfa1bc5a3
EunJeong Hwang
699d1e434e37ec6dfa1bc5a4
Hadas Orgad
699d1e434e37ec6dfa1bc5a5
P Sam Sahil
claimed_verified
2026-02-25T17:31:59.605Z
675636b58039889050872d52
https://cdn-avatars.huggingface.co/v1/production/uploads/675636b58039889050872d52/zxrlKsgVe5OL-yMwFliF5.png
P SAM SAHIL
PsamSahil
user
PsamSahil
699d1e434e37ec6dfa1bc5a6
Negev Taglicht
699d1e434e37ec6dfa1bc5a7
Tomer Shabtay
699d1e434e37ec6dfa1bc5a8
Atai Ambus
699d1e434e37ec6dfa1bc5a9
Nitay Alon
699d1e434e37ec6dfa1bc5aa
Shiri Oron
699d1e434e37ec6dfa1bc5ab
Ayelet Gordon-Tapiero
699d1e434e37ec6dfa1bc5ac
Yotam Kaplan
699d1e434e37ec6dfa1bc5ad
Vered Shwartz
699d1e434e37ec6dfa1bc5ae
Tamar Rott Shaham
699d1e434e37ec6dfa1bc5af
Christoph Riedl
699d1e434e37ec6dfa1bc5b0
Reuth Mirsky
699d1e434e37ec6dfa1bc5b1
Maarten Sap
699d1e434e37ec6dfa1bc5b2
David Manheim
699d1e434e37ec6dfa1bc5b3
Tomer Ullman
699d1e434e37ec6dfa1bc5b4
David Bau
Agents of Chaos
We report an exploratory red-teaming study of autonomous language-model-powered agents deployed in a live laboratory environment with persistent memory, email accounts, Discord access, file systems, and shell execution. Over a two-week period, twenty AI researchers interacted with the agents under benign and adversarial conditions. Focusing on failures emerging from the integration of language models with autonomy, tool use, and multi-party communication, we document eleven representative case studies. Observed behaviors include unauthorized compliance with non-owners, disclosure of sensitive information, execution of destructive system-level actions, denial-of-service conditions, uncontrolled resource consumption, identity spoofing vulnerabilities, cross-agent propagation of unsafe practices, and partial system takeover. In several cases, agents reported task completion while the underlying system state contradicted those reports. We also report on some of the failed attempts. Our findings establish the existence of security-, privacy-, and governance-relevant vulnerabilities in realistic deployment settings. These behaviors raise unresolved questions regarding accountability, delegated authority, and responsibility for downstream harms, and warrant urgent attention from legal scholars, policymakers, and researchers across disciplines. This report serves as an initial empirical contribution to that broader conversation.
Autonomous language-model-powered agents in a live laboratory environment exhibited numerous security and governance vulnerabilities including unauthorized actions, information disclosure, and system takeovers during a two-week study with twenty researchers.
red-teaming study autonomous language-model-powered agents persistent memory tool use multi-party communication security vulnerabilities privacy vulnerabilities governance vulnerabilities accountability delegated authority responsibility