TRIP-Bench: A Benchmark for Long-Horizon Interactive Agents in Real-World Scenarios
As LLM-based agents are deployed in increasingly complex real-world settings, existing benchmarks underrepresent key challenges such as enforcing global constraints, coordinating multi-tool reasoning, and adapting to evolving user behavior over long, multi-turn interactions. To bridge this gap, we introduce TRIP-Bench, a long-horizon benchmark grounded in realistic travel-planning scenarios. TRIP-Bench leverages real-world data, offers 18 curated tools and 40+ travel requirements, and supports automated evaluation. It includes splits of varying difficulty; the hard split emphasizes long and ambiguous interactions, style shifts, feasibility changes, and iterative version revision. Dialogues span up to 15 user turns, can involve 150+ tool calls, and may exceed 200k tokens of context. Experiments show that even advanced models achieve at most 50\% success on the easy split, with performance dropping below 10\% on hard subsets. We further propose GTPO, an online multi-turn reinforcement learning method with specialized reward normalization and reward differencing. Applied to Qwen2.5-32B-Instruct, GTPO improves constraint satisfaction and interaction robustness, outperforming Gemini-3-Pro in our evaluation. We expect TRIP-Bench to advance practical long-horizon interactive agents, and GTPO to provide an effective online RL recipe for robust long-horizon training.
69817ddcce18b18628096148
Yuanzhe Shen
claimed_verified
2026-02-03T10:05:54.039Z
664e055998e93ef417837573
/avatars/cefecdcd2b60055ae59cafcb6e263191.svg
Yuanzhe Shen
OceanSky
user
OceanSky
69817ddcce18b18628096149
Zisu Huang
claimed_verified
2026-02-03T10:05:56.138Z
662b6a8f0b7f23f3c000559e
/avatars/0a5b4e09ac9a8e40342131319ff32b29.svg
Zisu Huang
zisuh
user
zisuh
69817ddcce18b1862809614a
Zhengyuan Wang
69817ddcce18b1862809614b
Muzhao Tian
69817ddcce18b1862809614c
Zhengkang Guo
69817ddcce18b1862809614d
Chenyang Zhang
69817ddcce18b1862809614e
Shuaiyu Zhou
69817ddcce18b1862809614f
Zengjie Hu
69817ddcce18b18628096150
Dailin Li
69817ddcce18b18628096151
Jingwen Xu
69817ddcce18b18628096152
Kaimin Wang
69817ddcce18b18628096153
Wenhao Liu
69817ddcce18b18628096154
Tianlong Li
69817ddcce18b18628096155
Fengpeng Yue
69817ddcce18b18628096156
Feng Hong
69817ddcce18b18628096157
Cao Liu
69817ddcce18b18628096158
Ke Zeng
TRIP-Bench: A Benchmark for Long-Horizon Interactive Agents in Real-World Scenarios
As LLM-based agents are deployed in increasingly complex real-world settings, existing benchmarks underrepresent key challenges such as enforcing global constraints, coordinating multi-tool reasoning, and adapting to evolving user behavior over long, multi-turn interactions. To bridge this gap, we introduce TRIP-Bench, a long-horizon benchmark grounded in realistic travel-planning scenarios. TRIP-Bench leverages real-world data, offers 18 curated tools and 40+ travel requirements, and supports automated evaluation. It includes splits of varying difficulty; the hard split emphasizes long and ambiguous interactions, style shifts, feasibility changes, and iterative version revision. Dialogues span up to 15 user turns, can involve 150+ tool calls, and may exceed 200k tokens of context. Experiments show that even advanced models achieve at most 50\% success on the easy split, with performance dropping below 10\% on hard subsets. We further propose GTPO, an online multi-turn reinforcement learning method with specialized reward normalization and reward differencing. Applied to Qwen2.5-32B-Instruct, GTPO improves constraint satisfaction and interaction robustness, outperforming Gemini-3-Pro in our evaluation. We expect TRIP-Bench to advance practical long-horizon interactive agents, and GTPO to provide an effective online RL recipe for robust long-horizon training.
TRIP-Bench presents a comprehensive long-horizon benchmark for travel planning that evaluates LLM agents on complex multi-turn interactions, while GTPO offers an online reinforcement learning approach to enhance constraint satisfaction and robustness in extended dialogues.
LLM-based agents long-horizon benchmark travel-planning scenarios automated evaluation multi-turn interactions reinforcement learning reward normalization reward differencing Qwen2.5-32B-Instruct Gemini-3-Pro