DeepPlanning: Benchmarking Long-Horizon Agentic Planning with Verifiable Constraints
While agent evaluation has shifted toward long-horizon tasks, most benchmarks still emphasize local, step-level reasoning rather than the global constrained optimization (e.g., time and financial budgets) that demands genuine planning ability. Meanwhile, existing LLM planning benchmarks underrepresent the active information gathering and fine-grained local constraints typical of real-world settings. To address this, we introduce DeepPlanning, a challenging benchmark for practical long-horizon agent planning. It features multi-day travel planning and multi-product shopping tasks that require proactive information acquisition, local constrained reasoning, and global constrained optimization. Evaluations on DeepPlanning show that even frontier agentic LLMs struggle with these problems, highlighting the importance of reliable explicit reasoning patterns and parallel tool use for achieving better effectiveness-efficiency trade-offs. Error analysis further points to promising directions for improving agentic LLMs over long planning horizons. We open-source the code and data to support future research.
69784aee026bdf0473116f4e
Yinger Zhang
69784aee026bdf0473116f4f
Shutong Jiang
admin_assigned
2026-01-27T10:23:07.229Z
6746c6c99700a50f13a0eda9
/avatars/8e8b82a9b73ff807d976fc48bb2e3edc.svg
Shutong Jiang
Stjiang
user
Stjiang
69784aee026bdf0473116f50
Renhao Li
admin_assigned
2026-01-27T10:23:38.820Z
64abc87cf79cb0c313821c11
https://cdn-avatars.huggingface.co/v1/production/uploads/64abc87cf79cb0c313821c11/tJa4iX5_f5XoSIzXt-rwF.jpeg
Renhao Li
RioLee
user
RioLee
69784aee026bdf0473116f51
Jianhong Tu
admin_assigned
2026-01-27T10:23:44.994Z
654bead777401b47e6424f88
/avatars/7bcbdbb051c93b004f0dc3ad36c4a0ce.svg
Jianhong Tu
JianhongTu
user
JianhongTu
69784aee026bdf0473116f52
Yang Su
69784aee026bdf0473116f53
Lianghao Deng
69784aee026bdf0473116f54
Xudong Guo
69784aee026bdf0473116f55
Chenxu Lv
69784aee026bdf0473116f56
Junyang Lin
admin_assigned
2026-01-27T10:24:12.538Z
620760a26e3b7210c2ff1943
https://cdn-avatars.huggingface.co/v1/production/uploads/620760a26e3b7210c2ff1943/VC-rKqimF6yxGESNVlPoR.jpeg
Junyang Lin
JustinLin610
user
JustinLin610
DeepPlanning: Benchmarking Long-Horizon Agentic Planning with Verifiable Constraints
While agent evaluation has shifted toward long-horizon tasks, most benchmarks still emphasize local, step-level reasoning rather than the global constrained optimization (e.g., time and financial budgets) that demands genuine planning ability. Meanwhile, existing LLM planning benchmarks underrepresent the active information gathering and fine-grained local constraints typical of real-world settings. To address this, we introduce DeepPlanning, a challenging benchmark for practical long-horizon agent planning. It features multi-day travel planning and multi-product shopping tasks that require proactive information acquisition, local constrained reasoning, and global constrained optimization. Evaluations on DeepPlanning show that even frontier agentic LLMs struggle with these problems, highlighting the importance of reliable explicit reasoning patterns and parallel tool use for achieving better effectiveness-efficiency trade-offs. Error analysis further points to promising directions for improving agentic LLMs over long planning horizons. We open-source the code and data to support future research.
DeepPlanning benchmark addresses limitations of current LLM planning assessments by introducing complex, real-world tasks requiring both global optimization and local constraint reasoning.
agent evaluation long-horizon tasks global constrained optimization local constrained reasoning agentic LLMs explicit reasoning patterns parallel tool use