Beyond Accuracy: Unveiling Inefficiency Patterns in Tool-Integrated Reasoning
In real-world Tool-Integrated Reasoning (TIR) scenarios, where LLMs interleave reasoning with external tool calls, a major source of inefficiency is that the toolcalls create pauses between LLM requests and cause KV-Cache eviction, forcing recomputation. Also, the long, unfiltered response returned by external tools inflates the KV-Cache, so each decode step spends more time loading the growing cache and thus becomes steadily slower as context length increases. However, existing efficiency metrics like token counts and toolcall counts fail to capture the real model inference latency. To address this, we introduce PTE (Prefill Token Equivalents), a hardware-aware TIR-efficiency metric that unifies internal reasoning and external tool-use costs while explicitly accounting for non-reusable KV-Cache and long-tool-response scenarios. Validation in a high-concurrency industrial setting indicates that PTE aligns significantly better with wall-clock latency than standard token counts, while maintaining consistent efficiency rankings across diverse hardware profiles. We conduct extensive experiments across five TIR benchmarks, quantify their PTE costs, and identify four inefficiency patterns that appear in TIR. We also discover that trajectories with higher PTE costs tend to have lower reasoning correctness, indicating that simply using more tools does not improve the quality of the answer.
69d5b1d0b9b236b710ae8098
Qisheng Su
claimed_verified
2026-04-08T08:05:10.066Z
670a3bc3ada59c956f18cc17
/avatars/1a853deca3ec206f47aec2c6abe3a146.svg
SII-sqs
groundhogLLM
user
groundhogLLM
69d5b1d0b9b236b710ae8099
Shiting Huang
69d5b1d0b9b236b710ae809a
Zhen Fang
claimed_verified
2026-04-08T08:05:13.141Z
64b0a5037a475fba70a7260d
https://cdn-avatars.huggingface.co/v1/production/uploads/64b0a5037a475fba70a7260d/MauBbb6raMA23yrR1Zq21.jpeg
Zhen Fang
CostaliyA
user
CostaliyA
69d5b1d0b9b236b710ae809b
Ziyan Chen
claimed_verified
2026-04-09T08:07:18.149Z
64acaceecfb9ae8971d9927b
https://cdn-avatars.huggingface.co/v1/production/uploads/64acaceecfb9ae8971d9927b/A7qeBKoKDxEKGsU5x4skl.png
Ziyan Chen
CMander02
user
CMander02
69d5b1d0b9b236b710ae809c
Zehui Chen
69d5b1d0b9b236b710ae809d
Feng Zhao
Beyond Accuracy: Unveiling Inefficiency Patterns in Tool-Integrated Reasoning
In real-world Tool-Integrated Reasoning (TIR) scenarios, where LLMs interleave reasoning with external tool calls, a major source of inefficiency is that the toolcalls create pauses between LLM requests and cause KV-Cache eviction, forcing recomputation. Also, the long, unfiltered response returned by external tools inflates the KV-Cache, so each decode step spends more time loading the growing cache and thus becomes steadily slower as context length increases. However, existing efficiency metrics like token counts and toolcall counts fail to capture the real model inference latency. To address this, we introduce PTE (Prefill Token Equivalents), a hardware-aware TIR-efficiency metric that unifies internal reasoning and external tool-use costs while explicitly accounting for non-reusable KV-Cache and long-tool-response scenarios. Validation in a high-concurrency industrial setting indicates that PTE aligns significantly better with wall-clock latency than standard token counts, while maintaining consistent efficiency rankings across diverse hardware profiles. We conduct extensive experiments across five TIR benchmarks, quantify their PTE costs, and identify four inefficiency patterns that appear in TIR. We also discover that trajectories with higher PTE costs tend to have lower reasoning correctness, indicating that simply using more tools does not improve the quality of the answer.
Researchers introduce PTE (Prefill Token Equivalents), a hardware-aware metric for measuring efficiency in Tool-Integrated Reasoning scenarios, which better correlates with actual inference latency than traditional token counts by accounting for KV-Cache inefficiencies and long tool responses.
Tool-Integrated Reasoning LLMs KV-Cache eviction token counts toolcall counts hardware-aware metric prefill token equivalents wall-clock latency reasoning correctness