Learning When to Act or Refuse: Guarding Agentic Reasoning Models for Safe Multi-Step Tool Use
Agentic language models operate in a fundamentally different safety regime than chat models: they must plan, call tools, and execute long-horizon actions where a single misstep, such as accessing files or entering credentials, can cause irreversible harm. Existing alignment methods, largely optimized for static generation and task completion, break down in these settings due to sequential decision-making, adversarial tool feedback, and overconfident intermediate reasoning. We introduce MOSAIC, a post-training framework that aligns agents for safe multi-step tool use by making safety decisions explicit and learnable. MOSAIC structures inference as a plan, check, then act or refuse loop, with explicit safety reasoning and refusal as first-class actions. To train without trajectory-level labels, we use preference-based reinforcement learning with pairwise trajectory comparisons, which captures safety distinctions often missed by scalar rewards. We evaluate MOSAIC zero-shot across three model families, Qwen2.5-7B, Qwen3-4B-Thinking, and Phi-4, and across out-of-distribution benchmarks spanning harmful tasks, prompt injection, benign tool use, and cross-domain privacy leakage. MOSAIC reduces harmful behavior by up to 50%, increases harmful-task refusal by over 20% on injection attacks, cuts privacy leakage, and preserves or improves benign task performance, demonstrating robust generalization across models, domains, and agentic settings.
69a7ed9b2833ff1122ff664b
Aradhye Agarwal
claimed_verified
2026-03-06T13:56:57.545Z
63ca499104c97982831127ec
/avatars/816a962c62c805adf7789fa8e398b01e.svg
Aradhye Agarwal
aradhye
user
aradhye
69a7ed9b2833ff1122ff664c
Gurdit Siyan
69a7ed9b2833ff1122ff664d
Yash Pandya
claimed_verified
2026-03-05T09:59:41.786Z
669aa0cadc9c86e6ab733027
/avatars/dc4b5d04264ee69e10ea530186eaf338.svg
Yash Pandya
yashp96
user
yashp96
69a7ed9b2833ff1122ff664e
Joykirat Singh
69a7ed9b2833ff1122ff664f
Akshay Nambi
69a7ed9b2833ff1122ff6650
Ahmed Awadallah
Learning When to Act or Refuse: Guarding Agentic Reasoning Models for Safe Multi-Step Tool Use
Agentic language models operate in a fundamentally different safety regime than chat models: they must plan, call tools, and execute long-horizon actions where a single misstep, such as accessing files or entering credentials, can cause irreversible harm. Existing alignment methods, largely optimized for static generation and task completion, break down in these settings due to sequential decision-making, adversarial tool feedback, and overconfident intermediate reasoning. We introduce MOSAIC, a post-training framework that aligns agents for safe multi-step tool use by making safety decisions explicit and learnable. MOSAIC structures inference as a plan, check, then act or refuse loop, with explicit safety reasoning and refusal as first-class actions. To train without trajectory-level labels, we use preference-based reinforcement learning with pairwise trajectory comparisons, which captures safety distinctions often missed by scalar rewards. We evaluate MOSAIC zero-shot across three model families, Qwen2.5-7B, Qwen3-4B-Thinking, and Phi-4, and across out-of-distribution benchmarks spanning harmful tasks, prompt injection, benign tool use, and cross-domain privacy leakage. MOSAIC reduces harmful behavior by up to 50%, increases harmful-task refusal by over 20% on injection attacks, cuts privacy leakage, and preserves or improves benign task performance, demonstrating robust generalization across models, domains, and agentic settings.
MOSAIC is a post-training framework that aligns language model agents for safe multi-step tool use through explicit safety reasoning and refusal mechanisms, improving performance across diverse agentic scenarios.
post-training framework preference-based reinforcement learning pairwise trajectory comparisons safety reasoning refusal mechanisms agentic language models tool use trajectory-level labels scalar rewards benign task performance