CoMAS: Co-Evolving Multi-Agent Systems via Interaction Rewards
Self-evolution is a central research topic in enabling large language model
(LLM)-based agents to continually improve their capabilities after pretraining.
Recent research has witnessed a transition from reinforcement learning
(RL)-free to RL-based methods. Current RL-based methods either rely on dense
external reward signals or extract intrinsic reward signals from LLMs
themselves. However, these approaches diverge from the self-evolution
mechanisms observed in human intelligence, where individuals learn and improve
through mutual discussion and collaboration. In this work, we introduce
Co-Evolving Multi-Agent Systems (CoMAS), a novel framework that enables agents
to improve autonomously by learning from inter-agent interactions without
external supervision. CoMAS generates intrinsic rewards from rich discussion
dynamics, employs an LLM-as-a-judge mechanism to formulate these rewards, and
optimizes each agent's policy through RL, thereby enabling decentralized and
scalable co-evolution. Experimental results demonstrate that CoMAS consistently
outperforms untrained agents and achieves state-of-the-art performance across
most evaluation settings. Ablation studies confirm the necessity of
interaction-based reward signals and reveal promising scalability as the number
and diversity of agents increase. These findings establish CoMAS as a novel and
effective paradigm for self-evolution in LLM-based agents.
68e891e295e8e6771df38b41
Xiangyuan Xue
claimed_verified
2025-10-10T08:54:27.924Z
65082baabc8788c4064d5360
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/NZpFVnTpPGcCe8mvFMD-L.jpeg
Xiangyuan Xue
xxyQwQ
user
xxyQwQ
68e891e295e8e6771df38b42
Yifan Zhou
68e891e295e8e6771df38b43
Guibin Zhang
68e891e295e8e6771df38b44
Zaibin Zhang
claimed_verified
2026-01-21T09:26:04.282Z
6575702b15b1ca184b0b2700
https://cdn-avatars.huggingface.co/v1/production/uploads/6575702b15b1ca184b0b2700/O9cEodqQmG-gyqMiO_edR.jpeg
Zaibin Zhang
MrBean2024
user
MrBean2024
68e891e295e8e6771df38b45
Yijiang Li
claimed_verified
2026-02-11T11:22:05.416Z
6419309f22270b3ccf177c77
https://cdn-avatars.huggingface.co/v1/production/uploads/6419309f22270b3ccf177c77/KQa1586iBBKqucUlfpuPp.jpeg
William Li
williamium
user
williamium
68e891e295e8e6771df38b46
Chen Zhang
68e891e295e8e6771df38b47
Zhenfei Yin
68e891e295e8e6771df38b48
Philip Torr
68e891e295e8e6771df38b49
Wanli Ouyang
68e891e295e8e6771df38b4a
Lei Bai
CoMAS: Co-Evolving Multi-Agent Systems via Interaction Rewards
Self-evolution is a central research topic in enabling large language model
(LLM)-based agents to continually improve their capabilities after pretraining.
Recent research has witnessed a transition from reinforcement learning
(RL)-free to RL-based methods. Current RL-based methods either rely on dense
external reward signals or extract intrinsic reward signals from LLMs
themselves. However, these approaches diverge from the self-evolution
mechanisms observed in human intelligence, where individuals learn and improve
through mutual discussion and collaboration. In this work, we introduce
Co-Evolving Multi-Agent Systems (CoMAS), a novel framework that enables agents
to improve autonomously by learning from inter-agent interactions without
external supervision. CoMAS generates intrinsic rewards from rich discussion
dynamics, employs an LLM-as-a-judge mechanism to formulate these rewards, and
optimizes each agent's policy through RL, thereby enabling decentralized and
scalable co-evolution. Experimental results demonstrate that CoMAS consistently
outperforms untrained agents and achieves state-of-the-art performance across
most evaluation settings. Ablation studies confirm the necessity of
interaction-based reward signals and reveal promising scalability as the number
and diversity of agents increase. These findings establish CoMAS as a novel and
effective paradigm for self-evolution in LLM-based agents.
Co-Evolving Multi-Agent Systems (CoMAS) enable LLM-based agents to improve autonomously through inter-agent interactions and intrinsic rewards, achieving state-of-the-art performance.
self-evolution large language model (LLM) reinforcement learning (RL) intrinsic reward signals Co-Evolving Multi-Agent Systems (CoMAS) LLM-as-a-judge decentralized scalable co-evolution