MASPRM: Multi-Agent System Process Reward Model
Practical deployment of Multi-Agent Systems (MAS) demands strong test-time
performance, motivating methods that guide inference-time search and
selectively spend compute to improve quality. We present the Multi-Agent System
Process Reward Model (MASPRM). It assigns per-action, per-agent values to
partial inter-agent transcripts and acts as an inference-time controller.
MASPRM is trained from multi-agent Monte Carlo Tree Search (MCTS) rollouts
without requiring step-level human annotations, by propagating returns to local
targets. At inference, MASPRM guides step-level beam search and MCTS, focusing
computation on promising branches and pruning early. On GSM8K and MATH,
MASPRM-guided decoding with an outcome reward model (ORM) applied to the final
answer, improves exact match (EM) over a single straight-through MAS pass by
+30.7 and +22.9 points, respectively. A MASPRM trained on GSM8K transfers
zero-shot to MATH without retraining, adding 8.4 EM points at the same
budget. MASPRM is a plug-in value model that estimates per-agent progress and
complements verifier-style decoders, enabling more reliable, compute-aware
multi-agent reasoning. Code: https://github.com/milad1378yz/MASPRM
6902ae8672739622ee92a638
Milad Yazdani
claimed_verified
2025-10-30T14:32:59.839Z
624a82573571171191af5e60
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/tMIMn7oeSsDDpPuZ08co7.png
Milad Yazdani
Miladyz
user
Miladyz
6902ae8672739622ee92a639
Mahdi Mostajabdaveh
claimed_verified
2025-10-31T14:28:09.813Z
6521bd73dfc9ea4f31ad9f50
/avatars/b45ea369dbcbae4960e69ce53af11a46.svg
Mahdi M.
MahdiMs86
user
MahdiMs86
6902ae8672739622ee92a63a
Zirui Zhou
6902ae8672739622ee92a63b
Ying Xiong
MASPRM: Multi-Agent System Process Reward Model
Practical deployment of Multi-Agent Systems (MAS) demands strong test-time
performance, motivating methods that guide inference-time search and
selectively spend compute to improve quality. We present the Multi-Agent System
Process Reward Model (MASPRM). It assigns per-action, per-agent values to
partial inter-agent transcripts and acts as an inference-time controller.
MASPRM is trained from multi-agent Monte Carlo Tree Search (MCTS) rollouts
without requiring step-level human annotations, by propagating returns to local
targets. At inference, MASPRM guides step-level beam search and MCTS, focusing
computation on promising branches and pruning early. On GSM8K and MATH,
MASPRM-guided decoding with an outcome reward model (ORM) applied to the final
answer, improves exact match (EM) over a single straight-through MAS pass by
+30.7 and +22.9 points, respectively. A MASPRM trained on GSM8K transfers
zero-shot to MATH without retraining, adding 8.4 EM points at the same
budget. MASPRM is a plug-in value model that estimates per-agent progress and
complements verifier-style decoders, enabling more reliable, compute-aware
multi-agent reasoning. Code: https://github.com/milad1378yz/MASPRM
MASPRM, a process reward model for multi-agent systems, enhances inference performance by guiding beam search and MCTS, improving exact match scores on GSM8K and MATH datasets.
Multi-Agent Systems (MAS) inference-time controller Multi-Agent System Process Reward Model (MASPRM) multi-agent Monte Carlo Tree Search (MCTS) beam search MCTS outcome reward model (ORM) exact match (EM)