Collaborative Multi-Agent Test-Time Reinforcement Learning for Reasoning
Multi-agent systems have evolved into practical LLM-driven collaborators for many applications, gaining robustness from diversity and cross-checking. However, multi-agent RL (MARL) training is resource-intensive and unstable: co-adapting teammates induce non-stationarity, and rewards are often sparse and high-variance. Therefore, we introduce Multi-Agent Test-Time Reinforcement Learning (MATTRL), a framework that injects structured textual experience into multi-agent deliberation at inference time. MATTRL forms a multi-expert team of specialists for multi-turn discussions, retrieves and integrates test-time experiences, and reaches consensus for final decision-making. We also study credit assignment for constructing a turn-level experience pool, then reinjecting it into the dialogue. Across challenging benchmarks in medicine, math, and education, MATTRL improves accuracy by an average of 3.67\% over a multi-agent baseline, and by 8.67\% over comparable single-agent baselines. Ablation studies examine different credit-assignment schemes and provide a detailed comparison of how they affect training outcomes. MATTRL offers a stable, effective and efficient path to distribution-shift-robust multi-agent reasoning without tuning.
6969b0f732f0333869ff9476
Zhiyuan Hu
admin_assigned
2026-01-16T15:33:48.445Z
64351475901c5734bcb64248
/avatars/12346d4301c1bfb00ce0ea128a93cc15.svg
Zhiyuan Hu
zhiyuanhucs
user
zhiyuanhucs
6969b0f732f0333869ff9477
Yunhai Hu
claimed_verified
2026-01-16T14:37:06.706Z
662b4e3bc709a61df840fda1
/avatars/fc73c63a4e1f8fbb084ec43ec9af0af0.svg
Hu Yunhai
AlexCCtop
user
AlexCCtop
6969b0f732f0333869ff9478
Juncheng Liu
claimed_verified
2026-01-16T10:29:33.401Z
650026d30339dae3dba2cec5
/avatars/fcc9ea4336f8d4bb177e5c9eacdd05c9.svg
Juncheng Liu
juncliu
user
juncliu
6969b0f732f0333869ff9479
Shuyue Stella Li
6969b0f732f0333869ff947a
Yucheng Wang
claimed_verified
2026-01-26T08:33:53.054Z
6653f759b6316cf6f63fafec
/avatars/23770cd11e885305c3626d5596308770.svg
Yucheng Wang
Echoandland
user
Echoandland
6969b0f732f0333869ff947b
Zhen Xu
admin_assigned
2026-01-16T15:34:04.868Z
638e40d450a4e4beef98196b
/avatars/fe27e019baf48caeb44e19b7289db9fb.svg
Zhen Xu
zhenxu
user
zhenxu
6969b0f732f0333869ff947c
See-Kiong Ng
6969b0f732f0333869ff947d
Anh Tuan Luu
admin_assigned
2026-01-16T15:34:15.855Z
655722e80438e0854fae7554
/avatars/b93a74f7c7880f9fe0f3ffb47e2aef5e.svg
Luu Anh Tuan
anhtuanluu36
user
anhtuanluu36
6969b0f732f0333869ff947e
Xinxing Xu
6969b0f732f0333869ff947f
Bryan Hooi
admin_assigned
2026-01-16T15:34:25.577Z
651d8032c50012d33e914f2f
/avatars/0a44c9f51fc50ce86582e328c361ea00.svg
Bryan Hooi
bhooi
user
bhooi
6969b0f732f0333869ff9480
Cynthia Breazeal
admin_assigned
2026-01-16T15:34:31.289Z
672793ffa5255a517fd02045
/avatars/a2569be6f2e952b5b00e5d4b89a7cede.svg
Cynthia Breazeal
cynthiabreazeal
user
cynthiabreazeal
6969b0f732f0333869ff9481
Hae Won Park
admin_assigned
2026-01-16T15:34:36.481Z
682352cdb1c5350f850dd952
/avatars/5426efe0195ac8f914839e6585b1a112.svg
Hae Won Park
robohaewon
user
robohaewon
Collaborative Multi-Agent Test-Time Reinforcement Learning for Reasoning
Multi-agent systems have evolved into practical LLM-driven collaborators for many applications, gaining robustness from diversity and cross-checking. However, multi-agent RL (MARL) training is resource-intensive and unstable: co-adapting teammates induce non-stationarity, and rewards are often sparse and high-variance. Therefore, we introduce Multi-Agent Test-Time Reinforcement Learning (MATTRL), a framework that injects structured textual experience into multi-agent deliberation at inference time. MATTRL forms a multi-expert team of specialists for multi-turn discussions, retrieves and integrates test-time experiences, and reaches consensus for final decision-making. We also study credit assignment for constructing a turn-level experience pool, then reinjecting it into the dialogue. Across challenging benchmarks in medicine, math, and education, MATTRL improves accuracy by an average of 3.67\% over a multi-agent baseline, and by 8.67\% over comparable single-agent baselines. Ablation studies examine different credit-assignment schemes and provide a detailed comparison of how they affect training outcomes. MATTRL offers a stable, effective and efficient path to distribution-shift-robust multi-agent reasoning without tuning.
Multi-Agent Test-Time Reinforcement Learning (MATTRL) enhances multi-agent reasoning through structured textual experience injection and consensus-based decision making at inference time.
multi-agent systems reinforcement learning test-time reinforcement learning multi-agent reinforcement learning credit assignment multi-expert teams dialogue systems distribution-shift-robust reasoning