ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback
While LLM-based agents can interact with environments via invoking external tools, their expanded capabilities also amplify security risks. Monitoring step-level tool invocation behaviors in real time and proactively intervening before unsafe execution is critical for agent deployment, yet remains under-explored. In this work, we first construct TS-Bench, a novel benchmark for step-level tool invocation safety detection in LLM agents. We then develop a guardrail model, TS-Guard, using multi-task reinforcement learning. The model proactively detects unsafe tool invocation actions before execution by reasoning over the interaction history. It assesses request harmfulness and action-attack correlations, producing interpretable and generalizable safety judgments and feedback. Furthermore, we introduce TS-Flow, a guardrail-feedback-driven reasoning framework for LLM agents, which reduces harmful tool invocations of ReAct-style agents by 65 percent on average and improves benign task completion by approximately 10 percent under prompt injection attacks.
6969b57932f0333869ff9497
Yutao Mou
claimed_verified
2026-01-16T10:29:27.949Z
66632a3d2dc4dff9a98c38a5
https://cdn-avatars.huggingface.co/v1/production/uploads/66632a3d2dc4dff9a98c38a5/UYk99yT6M1mZviL8d2sm_.jpeg
Yutao Mou
MurrayTom
user
MurrayTom
6969b57932f0333869ff9498
Zhangchi Xue
6969b57932f0333869ff9499
Lijun Li
claimed_verified
2026-01-21T09:25:07.718Z
641d3efac3983aa9491677b9
/avatars/53565486351c16a1ac8ea863963e2d9b.svg
Lijun Li
adwardlee
user
adwardlee
6969b57932f0333869ff949a
Peiyang Liu
6969b57932f0333869ff949b
Shikun Zhang
6969b57932f0333869ff949c
Wei Ye
6969b57932f0333869ff949d
Jing Shao
ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback
While LLM-based agents can interact with environments via invoking external tools, their expanded capabilities also amplify security risks. Monitoring step-level tool invocation behaviors in real time and proactively intervening before unsafe execution is critical for agent deployment, yet remains under-explored. In this work, we first construct TS-Bench, a novel benchmark for step-level tool invocation safety detection in LLM agents. We then develop a guardrail model, TS-Guard, using multi-task reinforcement learning. The model proactively detects unsafe tool invocation actions before execution by reasoning over the interaction history. It assesses request harmfulness and action-attack correlations, producing interpretable and generalizable safety judgments and feedback. Furthermore, we introduce TS-Flow, a guardrail-feedback-driven reasoning framework for LLM agents, which reduces harmful tool invocations of ReAct-style agents by 65 percent on average and improves benign task completion by approximately 10 percent under prompt injection attacks.
A guardrail model and reasoning framework are developed to detect and prevent unsafe tool invocations in LLM agents, improving both safety and task performance under adversarial conditions.
LLM-based agents tool invocation safety detection multi-task reinforcement learning guardrail model ReAct-style agents prompt injection attacks