Are LLM Decisions Faithful to Verbal Confidence?
Large Language Models (LLMs) can produce surprisingly sophisticated estimates of their own uncertainty. However, it remains unclear to what extent this expressed confidence is tied to the reasoning, knowledge, or decision making of the model. To test this, we introduce RiskEval: a framework designed to evaluate whether models adjust their abstention policies in response to varying error penalties. Our evaluation of several frontier models reveals a critical dissociation: models are neither cost-aware when articulating their verbal confidence, nor strategically responsive when deciding whether to engage or abstain under high-penalty conditions. Even when extreme penalties render frequent abstention the mathematically optimal strategy, models almost never abstain, resulting in utility collapse. This indicates that calibrated verbal confidence scores may not be sufficient to create trustworthy and interpretable AI systems, as current models lack the strategic agency to convert uncertainty signals into optimal and risk-sensitive decisions.
6965e479fc8c4ecc02c7fa3d
Jiawei Wang
6965e479fc8c4ecc02c7fa3e
Yanfei Zhou
6965e479fc8c4ecc02c7fa3f
Siddartha Devic
6965e479fc8c4ecc02c7fa40
Deqing Fu
claimed_verified
2026-01-13T08:23:29.136Z
63c8454e46421a2efe82709d
https://cdn-avatars.huggingface.co/v1/production/uploads/63c8454e46421a2efe82709d/4k_HR5F6cxkVH1b5KEsGi.webp
Deqing Fu
deqing
user
deqing
Are LLM Decisions Faithful to Verbal Confidence?
Large Language Models (LLMs) can produce surprisingly sophisticated estimates of their own uncertainty. However, it remains unclear to what extent this expressed confidence is tied to the reasoning, knowledge, or decision making of the model. To test this, we introduce RiskEval: a framework designed to evaluate whether models adjust their abstention policies in response to varying error penalties. Our evaluation of several frontier models reveals a critical dissociation: models are neither cost-aware when articulating their verbal confidence, nor strategically responsive when deciding whether to engage or abstain under high-penalty conditions. Even when extreme penalties render frequent abstention the mathematically optimal strategy, models almost never abstain, resulting in utility collapse. This indicates that calibrated verbal confidence scores may not be sufficient to create trustworthy and interpretable AI systems, as current models lack the strategic agency to convert uncertainty signals into optimal and risk-sensitive decisions.
Large language models exhibit a disconnect between their expressed uncertainty and strategic decision-making under varying penalty conditions, failing to adjust abstention policies even when optimal.
large language models uncertainty estimation abstention policies risk-sensitive decisions calibration strategic agency