Code-Space Response Oracles: Generating Interpretable Multi-Agent Policies with Large Language Models
Recent advances in multi-agent reinforcement learning, particularly Policy-Space Response Oracles (PSRO), have enabled the computation of approximate game-theoretic equilibria in increasingly complex domains. However, these methods rely on deep reinforcement learning oracles that produce `black-box' neural network policies, making them difficult to interpret, trust or debug. We introduce Code-Space Response Oracles (CSRO), a novel framework that addresses this challenge by replacing RL oracles with Large Language Models (LLMs). CSRO reframes the best response computation as a code generation task, prompting an LLM to generate policies directly as human-readable code. This approach not only yields inherently interpretable policies but also leverages the LLM's pretrained knowledge to discover complex, human-like strategies. We explore multiple ways to construct and enhance an LLM-based oracle: zero-shot prompting, iterative refinement and AlphaEvolve, a distributed LLM-based evolutionary system. We demonstrate that CSRO achieves performance competitive with baselines while producing a diverse set of explainable policies. Our work presents a new perspective on multi-agent learning, shifting the focus from optimizing opaque policy parameters to synthesizing interpretable algorithmic behavior.
69b21d61206bad330cd7ff3e
Daniel Hennes
69b21d61206bad330cd7ff3f
Zun Li
69b21d61206bad330cd7ff40
John Schultz
69b21d61206bad330cd7ff41
Marc Lanctot
admin_assigned
2026-03-12T10:12:22.502Z
6781563e5fe908323d5a4ae9
/avatars/8dc486f4310e772e7f462726d57a3508.svg
Marc Lanctot
lanctot
user
lanctot
Code-Space Response Oracles: Generating Interpretable Multi-Agent Policies with Large Language Models
Recent advances in multi-agent reinforcement learning, particularly Policy-Space Response Oracles (PSRO), have enabled the computation of approximate game-theoretic equilibria in increasingly complex domains. However, these methods rely on deep reinforcement learning oracles that produce `black-box' neural network policies, making them difficult to interpret, trust or debug. We introduce Code-Space Response Oracles (CSRO), a novel framework that addresses this challenge by replacing RL oracles with Large Language Models (LLMs). CSRO reframes the best response computation as a code generation task, prompting an LLM to generate policies directly as human-readable code. This approach not only yields inherently interpretable policies but also leverages the LLM's pretrained knowledge to discover complex, human-like strategies. We explore multiple ways to construct and enhance an LLM-based oracle: zero-shot prompting, iterative refinement and AlphaEvolve, a distributed LLM-based evolutionary system. We demonstrate that CSRO achieves performance competitive with baselines while producing a diverse set of explainable policies. Our work presents a new perspective on multi-agent learning, shifting the focus from optimizing opaque policy parameters to synthesizing interpretable algorithmic behavior.
Code-Space Response Oracles replace traditional neural network policies with human-readable code generated by large language models, enabling interpretable and explainable multi-agent reinforcement learning.
multi-agent reinforcement learning Policy-Space Response Oracles large language models code generation interpretable policies AlphaEvolve evolutionary system