Read Your Own Mind: Reasoning Helps Surface Self-Confidence Signals in LLMs
This addresses the problem of unreliable self-confidence estimation in large language models for users needing trustworthy uncertainty quantification, though it's an incremental improvement on existing semantic entropy methods.
The researchers investigated why DeepSeek R1-32B shows overconfidence in its self-reported verbal confidence scores during question answering, finding that semantic entropy (requiring multiple response samples) provides more reliable uncertainty estimates. They demonstrated that forcing the model to generate long chain-of-thought reasoning before answering improves verbal confidence reliability by 25-40% on fact-retrieval tasks, and showed that a separate reader model can reconstruct similar confidences from just the reasoning chain.
We study the source of uncertainty in DeepSeek R1-32B by analyzing its self-reported verbal confidence on question answering (QA) tasks. In the default answer-then-confidence setting, the model is regularly over-confident, whereas semantic entropy - obtained by sampling many responses - remains reliable. We hypothesize that this is because of semantic entropy's larger test-time compute, which lets us explore the model's predictive distribution. We show that granting DeepSeek the budget to explore its distribution by forcing a long chain-of-thought before the final answer greatly improves its verbal score effectiveness, even on simple fact-retrieval questions that normally require no reasoning. Furthermore, a separate reader model that sees only the chain can reconstruct very similar confidences, indicating the verbal score might be merely a statistic of the alternatives surfaced during reasoning. Our analysis concludes that reliable uncertainty estimation requires explicit exploration of the generative space, and self-reported confidence is trustworthy only after such exploration.