ROI-Reasoning: Rational Optimization for Inference via Pre-Computation Meta-Cognition
Large language models (LLMs) can achieve strong reasoning performance with sufficient computation, but they do not inherently know how much computation a task requires. We study budgeted inference-time reasoning for multiple tasks under a strict global token constraint and formalize it as a Ordered Stochastic Multiple-Choice Knapsack Problem(OS-MCKP). This perspective highlights a meta-cognitive requirement -- anticipating task difficulty, estimating return over investment (ROI), and allocating computation strategically. We propose ROI-Reasoning, a two-stage framework that endows LLMs with intrinsic, budget-aware rationality. In the first stage, Meta-Cognitive Fine-Tuning teaches models to predict reasoning cost and expected utility before generation, enabling explicit solve-or-skip decisions. Next, Rationality-Aware Reinforcement Learning optimizes sequential decision making under a hard token budget, allowing models to learn long-horizon allocation strategies. Across budgeted mathematical reasoning benchmarks, ROI-Reasoning consistently improves overall score while substantially reducing regret under tight computation budgets.
695f21245fa3847525c41d2c
Muyang Zhao
claimed_verified
2026-01-08T08:32:09.136Z
673c429f4299bda07a4e2395
https://cdn-avatars.huggingface.co/v1/production/uploads/673c429f4299bda07a4e2395/T-dG_Hp5M6Vb2YpLobiD7.png
Sunny
wangwang318
user
wangwang318
695f21245fa3847525c41d2d
Qi Qi
695f21245fa3847525c41d2e
Hao Sun
ROI-Reasoning: Rational Optimization for Inference via Pre-Computation Meta-Cognition
Large language models (LLMs) can achieve strong reasoning performance with sufficient computation, but they do not inherently know how much computation a task requires. We study budgeted inference-time reasoning for multiple tasks under a strict global token constraint and formalize it as a Ordered Stochastic Multiple-Choice Knapsack Problem(OS-MCKP). This perspective highlights a meta-cognitive requirement -- anticipating task difficulty, estimating return over investment (ROI), and allocating computation strategically. We propose ROI-Reasoning, a two-stage framework that endows LLMs with intrinsic, budget-aware rationality. In the first stage, Meta-Cognitive Fine-Tuning teaches models to predict reasoning cost and expected utility before generation, enabling explicit solve-or-skip decisions. Next, Rationality-Aware Reinforcement Learning optimizes sequential decision making under a hard token budget, allowing models to learn long-horizon allocation strategies. Across budgeted mathematical reasoning benchmarks, ROI-Reasoning consistently improves overall score while substantially reducing regret under tight computation budgets.
Budgeted inference-time reasoning framework enables large language models to make strategic computational decisions by predicting costs and utilities before generation, then optimizing sequential allocation under strict token constraints through meta-cognitive fine-tuning and reinforcement learning.
large language models budgeted inference-time reasoning Ordered Stochastic Multiple-Choice Knapsack Problem meta-cognitive fine-tuning rationality-aware reinforcement learning reasoning cost expected utility solve-or-skip decisions sequential decision making token budget