Latent Collaboration in Multi-Agent Systems
Multi-agent systems (MAS) extend large language models (LLMs) from independent single-model reasoning to coordinative system-level intelligence. While existing LLM agents depend on text-based mediation for reasoning and communication, we take a step forward by enabling models to collaborate directly within the continuous latent space. We introduce LatentMAS, an end-to-end training-free framework that enables pure latent collaboration among LLM agents. In LatentMAS, each agent first performs auto-regressive latent thoughts generation through last-layer hidden embeddings. A shared latent working memory then preserves and transfers each agent's internal representations, ensuring lossless information exchange. We provide theoretical analyses establishing that LatentMAS attains higher expressiveness and lossless information preservation with substantially lower complexity than vanilla text-based MAS. In addition, empirical evaluations across 9 comprehensive benchmarks spanning math and science reasoning, commonsense understanding, and code generation show that LatentMAS consistently outperforms strong single-model and text-based MAS baselines, achieving up to 14.6% higher accuracy, reducing output token usage by 70.8%-83.7%, and providing 4x-4.3x faster end-to-end inference. These results demonstrate that our new latent collaboration framework enhances system-level reasoning quality while offering substantial efficiency gains without any additional training. Code and data are fully open-sourced at https://github.com/Gen-Verse/LatentMAS.
6927c504243b2216fb75cd62
Jiaru Zou
claimed_verified
2025-11-27T09:58:44.029Z
65c288280aa2d53135734a42
https://cdn-avatars.huggingface.co/v1/production/uploads/65c288280aa2d53135734a42/5WHmau52EaRS01TOMI3Qg.jpeg
Jiaru Zou
jiaruz2
user
jiaruz2
6927c504243b2216fb75cd63
Xiyuan Yang
6927c504243b2216fb75cd64
Ruizhong Qiu
6927c504243b2216fb75cd65
Gaotang Li
6927c504243b2216fb75cd66
Katherine Tieu
6927c504243b2216fb75cd67
Pan Lu
claimed_verified
2025-11-27T09:58:40.924Z
60f5f68fa7fd83d025749234
https://cdn-avatars.huggingface.co/v1/production/uploads/60f5f68fa7fd83d025749234/gCeJAZfzaANAcEvI6v5-P.jpeg
Pan Lu
lupantech
user
lupantech
6927c504243b2216fb75cd68
Ke Shen
6927c504243b2216fb75cd69
Hanghang Tong
6927c504243b2216fb75cd6a
Yejin Choi
6927c504243b2216fb75cd6b
Jingrui He
6927c504243b2216fb75cd6c
James Zou
6927c504243b2216fb75cd6d
Mengdi Wang
6927c504243b2216fb75cd6e
Ling Yang
claimed_verified
2025-11-28T13:48:57.250Z
64fde4e252e82dd432b74ce9
https://cdn-avatars.huggingface.co/v1/production/uploads/64fde4e252e82dd432b74ce9/-CQZbBP7FsPPyawYrsi4z.jpeg
Ling Yang
Lingaaaaaaa
user
Lingaaaaaaa
Latent Collaboration in Multi-Agent Systems
Multi-agent systems (MAS) extend large language models (LLMs) from independent single-model reasoning to coordinative system-level intelligence. While existing LLM agents depend on text-based mediation for reasoning and communication, we take a step forward by enabling models to collaborate directly within the continuous latent space. We introduce LatentMAS, an end-to-end training-free framework that enables pure latent collaboration among LLM agents. In LatentMAS, each agent first performs auto-regressive latent thoughts generation through last-layer hidden embeddings. A shared latent working memory then preserves and transfers each agent's internal representations, ensuring lossless information exchange. We provide theoretical analyses establishing that LatentMAS attains higher expressiveness and lossless information preservation with substantially lower complexity than vanilla text-based MAS. In addition, empirical evaluations across 9 comprehensive benchmarks spanning math and science reasoning, commonsense understanding, and code generation show that LatentMAS consistently outperforms strong single-model and text-based MAS baselines, achieving up to 14.6% higher accuracy, reducing output token usage by 70.8%-83.7%, and providing 4x-4.3x faster end-to-end inference. These results demonstrate that our new latent collaboration framework enhances system-level reasoning quality while offering substantial efficiency gains without any additional training. Code and data are fully open-sourced at https://github.com/Gen-Verse/LatentMAS.
LatentMAS enables efficient, lossless collaboration among LLM agents in latent space, improving performance and reducing computational costs compared to text-based methods.
multi-agent systems large language models latent space LatentMAS auto-regressive latent thoughts generation latent working memory expressiveness information exchange math and science reasoning commonsense understanding code generation end-to-end inference