Combee: Scaling Prompt Learning for Self-Improving Language Model Agents
Recent advances in prompt learning allow large language model agents to acquire task-relevant knowledge from inference-time context without parameter changes. For example, existing methods (like ACE or GEPA) can learn system prompts to improve accuracy based on previous agent runs. However, these methods primarily focus on single-agent or low-parallelism settings. This fundamentally limits their ability to efficiently learn from a large set of collected agentic traces. It would be efficient and beneficial to run prompt learning in parallel to accommodate the growing trend of learning from many agentic traces or parallel agent executions. Yet without a principled strategy for scaling, current methods suffer from quality degradation with high parallelism. To improve both the efficiency and quality of prompt learning, we propose Combee, a novel framework to scale parallel prompt learning for self-improving agents. Combee speeds up learning and enables running many agents in parallel while learning from their aggregate traces without quality degradation. To achieve this, Combee leverages parallel scans and employs an augmented shuffle mechanism; Combee also introduces a dynamic batch size controller to balance quality and delay. Evaluations on AppWorld, Terminal-Bench, Formula, and FiNER demonstrate that Combee achieves up to 17x speedup over previous methods with comparable or better accuracy and equivalent cost.
69d52cb963f089495f06b29a
Hanchen Li
69d52cb963f089495f06b29b
Runyuan He
69d52cb963f089495f06b29c
Qizheng Zhang
claimed_verified
2026-04-13T07:34:53.441Z
6663e5b10c54dffcd2a921ca
/avatars/2a4589fef05306ccde06728c752e5601.svg
Qizheng Zhang
qizhengz
user
qizhengz
69d52cb963f089495f06b29d
Changxiu Ji
69d52cb963f089495f06b29e
Qiuyang Mang
69d52cb963f089495f06b29f
Xiaokun Chen
69d52cb963f089495f06b2a0
Lakshya A Agrawal
69d52cb963f089495f06b2a1
Wei-Liang Liao
69d52cb963f089495f06b2a2
Eric Yang
69d52cb963f089495f06b2a3
Alvin Cheung
69d52cb963f089495f06b2a4
James Zou
69d52cb963f089495f06b2a5
Kunle Olukotun
69d52cb963f089495f06b2a6
Ion Stoica
69d52cb963f089495f06b2a7
Joseph E. Gonzalez
Combee: Scaling Prompt Learning for Self-Improving Language Model Agents
Recent advances in prompt learning allow large language model agents to acquire task-relevant knowledge from inference-time context without parameter changes. For example, existing methods (like ACE or GEPA) can learn system prompts to improve accuracy based on previous agent runs. However, these methods primarily focus on single-agent or low-parallelism settings. This fundamentally limits their ability to efficiently learn from a large set of collected agentic traces. It would be efficient and beneficial to run prompt learning in parallel to accommodate the growing trend of learning from many agentic traces or parallel agent executions. Yet without a principled strategy for scaling, current methods suffer from quality degradation with high parallelism. To improve both the efficiency and quality of prompt learning, we propose Combee, a novel framework to scale parallel prompt learning for self-improving agents. Combee speeds up learning and enables running many agents in parallel while learning from their aggregate traces without quality degradation. To achieve this, Combee leverages parallel scans and employs an augmented shuffle mechanism; Combee also introduces a dynamic batch size controller to balance quality and delay. Evaluations on AppWorld, Terminal-Bench, Formula, and FiNER demonstrate that Combee achieves up to 17x speedup over previous methods with comparable or better accuracy and equivalent cost.
Combee is a framework that enables scalable parallel prompt learning for self-improving agents by using parallel scans, augmented shuffle mechanisms, and dynamic batch size control to maintain quality while achieving significant speedups.
prompt learning large language model agents system prompts agentic traces parallel prompt learning self-improving agents parallel scans augmented shuffle mechanism dynamic batch size controller