KARL: Knowledge Agents via Reinforcement Learning
We present a system for training enterprise search agents via reinforcement learning that achieves state-of-the-art performance across a diverse suite of hard-to-verify agentic search tasks. Our work makes four core contributions. First, we introduce KARLBench, a multi-capability evaluation suite spanning six distinct search regimes, including constraint-driven entity search, cross-document report synthesis, tabular numerical reasoning, exhaustive entity retrieval, procedural reasoning over technical documentation, and fact aggregation over internal enterprise notes. Second, we show that models trained across heterogeneous search behaviors generalize substantially better than those optimized for any single benchmark. Third, we develop an agentic synthesis pipeline that employs long-horizon reasoning and tool use to generate diverse, grounded, and high-quality training data, with iterative bootstrapping from increasingly capable models. Fourth, we propose a new post-training paradigm based on iterative large-batch off-policy RL that is sample efficient, robust to train-inference engine discrepancies, and naturally extends to multi-task training with out-of-distribution generalization. Compared to Claude 4.6 and GPT 5.2, KARL is Pareto-optimal on KARLBench across cost-quality and latency-quality trade-offs, including tasks that were out-of-distribution during training. With sufficient test-time compute, it surpasses the strongest closed models. These results show that tailored synthetic data in combination with multi-task reinforcement learning enables cost-efficient and high-performing knowledge agents for grounded reasoning.
69aa456bc4f6edddcc6c8607
Jonathan D. Chang
69aa456bc4f6edddcc6c8608
Andrew Drozdov
69aa456bc4f6edddcc6c8609
Shubham Toshniwal
69aa456bc4f6edddcc6c860a
Owen Oertell
69aa456bc4f6edddcc6c860b
Alexander Trott
69aa456bc4f6edddcc6c860c
Jacob Portes
69aa456bc4f6edddcc6c860d
Abhay Gupta
69aa456bc4f6edddcc6c860e
Pallavi Koppol
69aa456bc4f6edddcc6c860f
Ashutosh Baheti
69aa456bc4f6edddcc6c8610
Sean Kulinski
69aa456bc4f6edddcc6c8611
Ivan Zhou
69aa456bc4f6edddcc6c8612
Irene Dea
69aa456bc4f6edddcc6c8613
Krista Opsahl-Ong
69aa456bc4f6edddcc6c8614
Simon Favreau-Lessard
69aa456bc4f6edddcc6c8615
Sean Owen
69aa456bc4f6edddcc6c8616
Jose Javier Gonzalez Ortiz
69aa456bc4f6edddcc6c8617
Arnav Singhvi
69aa456bc4f6edddcc6c8618
Xabi Andrade
69aa456bc4f6edddcc6c8619
Cindy Wang
69aa456bc4f6edddcc6c861a
Kartik Sreenivasan
69aa456bc4f6edddcc6c861b
Sam Havens
69aa456bc4f6edddcc6c861c
Jialu Liu
69aa456bc4f6edddcc6c861d
Peyton DeNiro
69aa456bc4f6edddcc6c861e
Wen Sun
69aa456bc4f6edddcc6c861f
Michael Bendersky
69aa456bc4f6edddcc6c8620
Jonathan Frankle
KARL: Knowledge Agents via Reinforcement Learning
We present a system for training enterprise search agents via reinforcement learning that achieves state-of-the-art performance across a diverse suite of hard-to-verify agentic search tasks. Our work makes four core contributions. First, we introduce KARLBench, a multi-capability evaluation suite spanning six distinct search regimes, including constraint-driven entity search, cross-document report synthesis, tabular numerical reasoning, exhaustive entity retrieval, procedural reasoning over technical documentation, and fact aggregation over internal enterprise notes. Second, we show that models trained across heterogeneous search behaviors generalize substantially better than those optimized for any single benchmark. Third, we develop an agentic synthesis pipeline that employs long-horizon reasoning and tool use to generate diverse, grounded, and high-quality training data, with iterative bootstrapping from increasingly capable models. Fourth, we propose a new post-training paradigm based on iterative large-batch off-policy RL that is sample efficient, robust to train-inference engine discrepancies, and naturally extends to multi-task training with out-of-distribution generalization. Compared to Claude 4.6 and GPT 5.2, KARL is Pareto-optimal on KARLBench across cost-quality and latency-quality trade-offs, including tasks that were out-of-distribution during training. With sufficient test-time compute, it surpasses the strongest closed models. These results show that tailored synthetic data in combination with multi-task reinforcement learning enables cost-efficient and high-performing knowledge agents for grounded reasoning.
A reinforcement learning system for enterprise search agents achieves superior performance through diverse training data generation and multi-task learning approaches.
reinforcement learning agentic search multi-task training off-policy RL synthetic data generation long-horizon reasoning tool use iterative bootstrapping post-training paradigm sample efficiency out-of-distribution generalization Pareto-optimal