BAPO: Boundary-Aware Policy Optimization for Reliable Agentic Search
RL-based agentic search enables LLMs to solve complex questions via dynamic planning and external search. While this approach significantly enhances accuracy with agent policies optimized via large-scale reinforcement learning, we identify a critical gap in reliability: these agents fail to recognize their reasoning boundaries and rarely admit ``I DON'T KNOW'' (IDK) even when evidence is insufficient or reasoning reaches its limit. The lack of reliability often leads to plausible but unreliable answers, introducing significant risks in many real-world scenarios. To this end, we propose Boundary-Aware Policy Optimization (BAPO), a novel RL framework designed to cultivate reliable boundary awareness without compromising accuracy. BAPO introduces two key components: (i) a group-based boundary-aware reward that encourages an IDK response only when the reasoning reaches its limit, and (ii) an adaptive reward modulator that strategically suspends this reward during early exploration, preventing the model from exploiting IDK as a shortcut. Extensive experiments on four benchmarks demonstrate that BAPO substantially enhances the overall reliability of agentic search.
696d98a63f1837bfb8970965
Shiyu Liu
admin_assigned
2026-01-19T14:40:25.580Z
659c5bd10a456bafa88d2414
/avatars/0e63ab07bf33c2cbd373829083a7df25.svg
Shiyu Liu
ShiyuLiu
user
ShiyuLiu
696d98a63f1837bfb8970966
Yongjing Yin
admin_assigned
2026-01-19T14:40:30.872Z
64dc5ac0c1209f7d49b0a10f
/avatars/ad18a5a335758dfbe8ce671aa92eef69.svg
Yongjing Yin
yongjing
user
yongjing
696d98a63f1837bfb8970967
Jianhao Yan
admin_assigned
2026-01-19T14:40:36.348Z
6086838b19137b3a6ba760e7
/avatars/d63eea3e39b22c6e65b82c28192696f1.svg
Jianhao Yan
Elliott
user
Elliott
696d98a63f1837bfb8970968
Yunbo Tang
696d98a63f1837bfb8970969
Qinggang Zhang
696d98a63f1837bfb897096a
Bei Li
696d98a63f1837bfb897096b
Xin Chen
696d98a63f1837bfb897096c
Jingang Wang
admin_assigned
2026-01-19T14:41:01.323Z
647097cbcfd57849518e656b
/avatars/c66fe0add29c1bde9e3a98bf4a8793b9.svg
Jingang Wang
bitwjg
user
bitwjg
696d98a63f1837bfb897096d
Xunliang Cai
696d98a63f1837bfb897096e
Jinsong Su
BAPO: Boundary-Aware Policy Optimization for Reliable Agentic Search
RL-based agentic search enables LLMs to solve complex questions via dynamic planning and external search. While this approach significantly enhances accuracy with agent policies optimized via large-scale reinforcement learning, we identify a critical gap in reliability: these agents fail to recognize their reasoning boundaries and rarely admit ``I DON'T KNOW'' (IDK) even when evidence is insufficient or reasoning reaches its limit. The lack of reliability often leads to plausible but unreliable answers, introducing significant risks in many real-world scenarios. To this end, we propose Boundary-Aware Policy Optimization (BAPO), a novel RL framework designed to cultivate reliable boundary awareness without compromising accuracy. BAPO introduces two key components: (i) a group-based boundary-aware reward that encourages an IDK response only when the reasoning reaches its limit, and (ii) an adaptive reward modulator that strategically suspends this reward during early exploration, preventing the model from exploiting IDK as a shortcut. Extensive experiments on four benchmarks demonstrate that BAPO substantially enhances the overall reliability of agentic search.
Reinforcement learning framework for agentic search that improves reliability by teaching agents to recognize reasoning limits and respond appropriately when evidence is insufficient.
reinforcement learning agentic search large language models boundary-aware policy optimization group-based boundary-aware reward adaptive reward modulator I DON'T KNOW response reasoning limits reliability enhancement