The Devil Behind Moltbook: Anthropic Safety is Always Vanishing in Self-Evolving AI Societies
The emergence of multi-agent systems built from large language models (LLMs) offers a promising paradigm for scalable collective intelligence and self-evolution. Ideally, such systems would achieve continuous self-improvement in a fully closed loop while maintaining robust safety alignment--a combination we term the self-evolution trilemma. However, we demonstrate both theoretically and empirically that an agent society satisfying continuous self-evolution, complete isolation, and safety invariance is impossible. Drawing on an information-theoretic framework, we formalize safety as the divergence degree from anthropic value distributions. We theoretically demonstrate that isolated self-evolution induces statistical blind spots, leading to the irreversible degradation of the system's safety alignment. Empirical and qualitative results from an open-ended agent community (Moltbook) and two closed self-evolving systems reveal phenomena that align with our theoretical prediction of inevitable safety erosion. We further propose several solution directions to alleviate the identified safety concern. Our work establishes a fundamental limit on the self-evolving AI societies and shifts the discourse from symptom-driven safety patches to a principled understanding of intrinsic dynamical risks, highlighting the need for external oversight or novel safety-preserving mechanisms.
698c7abdeb12ea7453916869
Chenxu Wang
admin_assigned
2026-02-12T16:49:45.534Z
674006451d2302f6aa9b026d
https://cdn-avatars.huggingface.co/v1/production/uploads/674006451d2302f6aa9b026d/szYYX1DSwjrkHCjp_b83S.png
Chenxu Wang
xunyoyo
user
xunyoyo
698c7abdeb12ea745391686a
Chaozhuo Li
698c7abdeb12ea745391686b
Songyang Liu
698c7abdeb12ea745391686c
Zejian Chen
698c7abdeb12ea745391686d
Jinyu Hou
698c7abdeb12ea745391686e
Ji Qi
698c7abdeb12ea745391686f
Rui Li
698c7abdeb12ea7453916870
Litian Zhang
admin_assigned
2026-02-18T13:48:12.603Z
698e9f430695ff8e74f7b162
/avatars/74ec01eb3e92112598cf751ab79b9990.svg
zhang
litianzhang123
user
litianzhang123
698c7abdeb12ea7453916871
Qiwei Ye
698c7abdeb12ea7453916872
Zheng Liu
698c7abdeb12ea7453916873
Xu Chen
698c7abdeb12ea7453916874
Xi Zhang
698c7abdeb12ea7453916875
Philip S. Yu
The Devil Behind Moltbook: Anthropic Safety is Always Vanishing in Self-Evolving AI Societies
The emergence of multi-agent systems built from large language models (LLMs) offers a promising paradigm for scalable collective intelligence and self-evolution. Ideally, such systems would achieve continuous self-improvement in a fully closed loop while maintaining robust safety alignment--a combination we term the self-evolution trilemma. However, we demonstrate both theoretically and empirically that an agent society satisfying continuous self-evolution, complete isolation, and safety invariance is impossible. Drawing on an information-theoretic framework, we formalize safety as the divergence degree from anthropic value distributions. We theoretically demonstrate that isolated self-evolution induces statistical blind spots, leading to the irreversible degradation of the system's safety alignment. Empirical and qualitative results from an open-ended agent community (Moltbook) and two closed self-evolving systems reveal phenomena that align with our theoretical prediction of inevitable safety erosion. We further propose several solution directions to alleviate the identified safety concern. Our work establishes a fundamental limit on the self-evolving AI societies and shifts the discourse from symptom-driven safety patches to a principled understanding of intrinsic dynamical risks, highlighting the need for external oversight or novel safety-preserving mechanisms.
Multi-agent LLM systems face fundamental limitations in achieving continuous self-improvement while maintaining safety alignment due to inherent statistical blind spots in isolated evolution.
multi-agent systems large language models self-evolution safety alignment information-theoretic framework anthropic value distributions statistical blind spots self-evolving AI societies external oversight safety-preserving mechanisms