Spend Less, Reason Better: Budget-Aware Value Tree Search for LLM Agents
Test-time scaling has become a dominant paradigm for improving LLM agent reliability, yet current approaches treat compute as an abundant resource, allowing agents to exhaust token and tool budgets on redundant steps or dead-end trajectories. Existing budget-aware methods either require expensive fine-tuning or rely on coarse, trajectory-level heuristics that cannot intervene mid-execution. We propose the Budget-Aware Value Tree (BAVT), a training-free inference-time framework that models multi-hop reasoning as a dynamic search tree guided by step-level value estimation within a single LLM backbone. Another key innovation is a budget-conditioned node selection mechanism that uses the remaining resource ratio as a natural scaling exponent over node values, providing a principled, parameter-free transition from broad exploration to greedy exploitation as the budget depletes. To combat the well-known overconfidence of LLM self-evaluation, BAVT employs a residual value predictor that scores relative progress rather than absolute state quality, enabling reliable pruning of uninformative or redundant tool calls. We further provide a theoretical convergence guarantee, proving that BAVT reaches a terminal answer with probability at least 1-ε under an explicit finite budget bound. Extensive evaluations on four multi-hop QA benchmarks across two model families demonstrate that BAVT consistently outperforms parallel sampling baselines. Most notably, BAVT under strict low-budget constraints surpasses baseline performance at 4times the resource allocation, establishing that intelligent budget management fundamentally outperforms brute-force compute scaling.
69b7633d91a28162b233fbf2
Yushu Li
admin_assigned
2026-03-16T11:12:16.529Z
66c395e1b83a7e94d55f9ca1
/avatars/01fb36b3db43d3f36ba352d49d966410.svg
Yushu Li
yushuli
user
yushuli
69b7633d91a28162b233fbf3
Wenlong Deng
admin_assigned
2026-03-16T11:12:09.561Z
663256cc87ce9a8935cb1318
/avatars/f7c4a07dd026497f3b134db20540193c.svg
wenlong deng
dwenlong
user
dwenlong
69b7633d91a28162b233fbf4
Jiajin Li
69b7633d91a28162b233fbf5
Xiaoxiao Li
Spend Less, Reason Better: Budget-Aware Value Tree Search for LLM Agents
Test-time scaling has become a dominant paradigm for improving LLM agent reliability, yet current approaches treat compute as an abundant resource, allowing agents to exhaust token and tool budgets on redundant steps or dead-end trajectories. Existing budget-aware methods either require expensive fine-tuning or rely on coarse, trajectory-level heuristics that cannot intervene mid-execution. We propose the Budget-Aware Value Tree (BAVT), a training-free inference-time framework that models multi-hop reasoning as a dynamic search tree guided by step-level value estimation within a single LLM backbone. Another key innovation is a budget-conditioned node selection mechanism that uses the remaining resource ratio as a natural scaling exponent over node values, providing a principled, parameter-free transition from broad exploration to greedy exploitation as the budget depletes. To combat the well-known overconfidence of LLM self-evaluation, BAVT employs a residual value predictor that scores relative progress rather than absolute state quality, enabling reliable pruning of uninformative or redundant tool calls. We further provide a theoretical convergence guarantee, proving that BAVT reaches a terminal answer with probability at least 1-ε under an explicit finite budget bound. Extensive evaluations on four multi-hop QA benchmarks across two model families demonstrate that BAVT consistently outperforms parallel sampling baselines. Most notably, BAVT under strict low-budget constraints surpasses baseline performance at 4times the resource allocation, establishing that intelligent budget management fundamentally outperforms brute-force compute scaling.
A budget-aware value tree framework enables efficient multi-hop reasoning in language models by dynamically balancing exploration and exploitation based on remaining computational resources.
test-time scaling LLM agent reliability compute resource management multi-hop reasoning dynamic search tree step-level value estimation budget-conditioned node selection residual value predictor theoretical convergence guarantee parallel sampling baselines