Scaling Agentic Capabilities, Not Context: Efficient Reinforcement Finetuning for Large Toolspaces
Agentic systems operating over large tool ecosystems must plan and execute long-horizon workflows under weak or non-verifiable supervision. While frontier models mitigate these challenges through scale and large context budgets, small language models (SLMs) remain brittle: eager tool loading saturates context, execution errors compound over time, and sparse rewards limit learning. We introduce ATLAS, a reinforcement finetuning framework that enables SLMs to operate effectively in large-scale toolspace environments by learning how to acquire context and how to execute actions. Our approach makes two key contributions. First, we treat context control and execution structure as learnable decisions, combining iterative tool loading with programmatic tool orchestration to bound context growth and stabilize long-horizon trajectories. Second, we propose rubric-based reinforcement finetuning, which decomposes task success into structured, task-aligned criteria and enables scalable training using small judge models. Across MCP benchmarks, these design choices yield large and consistent gains over generic RL baselines, allowing a 4B SLM to approach frontier-agent performance under far tighter parameter and context budgets.
69af947bb3ab31dee1281d44
Karan Gupta
69af947bb3ab31dee1281d45
Pranav Vajreshwari
69af947bb3ab31dee1281d46
Yash Pandya
claimed_verified
2026-03-10T09:44:57.996Z
669aa0cadc9c86e6ab733027
/avatars/dc4b5d04264ee69e10ea530186eaf338.svg
Yash Pandya
yashp96
user
yashp96
69af947bb3ab31dee1281d47
Raghav Magazine
69af947bb3ab31dee1281d48
Akshay Nambi
69af947bb3ab31dee1281d49
Ahmed Awadallah
Scaling Agentic Capabilities, Not Context: Efficient Reinforcement Finetuning for Large Toolspaces
Agentic systems operating over large tool ecosystems must plan and execute long-horizon workflows under weak or non-verifiable supervision. While frontier models mitigate these challenges through scale and large context budgets, small language models (SLMs) remain brittle: eager tool loading saturates context, execution errors compound over time, and sparse rewards limit learning. We introduce ATLAS, a reinforcement finetuning framework that enables SLMs to operate effectively in large-scale toolspace environments by learning how to acquire context and how to execute actions. Our approach makes two key contributions. First, we treat context control and execution structure as learnable decisions, combining iterative tool loading with programmatic tool orchestration to bound context growth and stabilize long-horizon trajectories. Second, we propose rubric-based reinforcement finetuning, which decomposes task success into structured, task-aligned criteria and enables scalable training using small judge models. Across MCP benchmarks, these design choices yield large and consistent gains over generic RL baselines, allowing a 4B SLM to approach frontier-agent performance under far tighter parameter and context budgets.
ATLAS enables small language models to effectively operate in large-scale tool environments through reinforcement fine-tuning that learns context control and execution structure, achieving performance comparable to larger models under stricter resource constraints.
reinforcement fine-tuning context control execution structure tool orchestration programmatic tool orchestration rubric-based reinforcement fine-tuning task-aligned criteria agent performance parameter efficiency context budget