The Art of Scaling Test-Time Compute for Large Language Models
Test-time scaling (TTS) -- the dynamic allocation of compute during inference -- is a promising direction for improving reasoning in large language models (LLMs). However, a systematic comparison of well-known TTS strategies under identical conditions is missing, and the influence of model type and problem difficulty on performance remains unclear. To address these gaps, we conduct the first large-scale study of TTS, spanning over thirty billion tokens generated using eight open-source LLMs (7B to 235B parameters), across four reasoning datasets. We observe three consistent trends: (1) no single TTS strategy universally dominates; (2) reasoning models exhibit distinct trace-quality patterns across problem difficulty and trace length, forming short-horizon and long-horizon categories; and (3) for a given model type, the optimal TTS performance scales monotonically with compute budget. Based on these insights, we provide a practical recipe for selecting the best TTS strategy, considering problem difficulty, model type, and compute budget, providing a practical guide to effective inference-time scaling.
692e749937312eaa83fd88c4
Aradhye Agarwal
claimed_verified
2026-03-06T13:58:41.462Z
63ca499104c97982831127ec
/avatars/816a962c62c805adf7789fa8e398b01e.svg
Aradhye Agarwal
aradhye
user
aradhye
692e749937312eaa83fd88c5
Ayan Sengupta
692e749937312eaa83fd88c6
Tanmoy Chakraborty
The Art of Scaling Test-Time Compute for Large Language Models
Test-time scaling (TTS) -- the dynamic allocation of compute during inference -- is a promising direction for improving reasoning in large language models (LLMs). However, a systematic comparison of well-known TTS strategies under identical conditions is missing, and the influence of model type and problem difficulty on performance remains unclear. To address these gaps, we conduct the first large-scale study of TTS, spanning over thirty billion tokens generated using eight open-source LLMs (7B to 235B parameters), across four reasoning datasets. We observe three consistent trends: (1) no single TTS strategy universally dominates; (2) reasoning models exhibit distinct trace-quality patterns across problem difficulty and trace length, forming short-horizon and long-horizon categories; and (3) for a given model type, the optimal TTS performance scales monotonically with compute budget. Based on these insights, we provide a practical recipe for selecting the best TTS strategy, considering problem difficulty, model type, and compute budget, providing a practical guide to effective inference-time scaling.
Systematic study of test-time scaling strategies in large language models reveals distinct performance trends based on problem difficulty, model type, and compute budget.
test-time scaling TTS dynamic allocation inference large language models LLMs trace-quality patterns short-horizon long-horizon