Causal Evidence that Language Models use Confidence to Drive Behavior
This addresses the fundamental question of metacognition in LLMs, which is essential for their transition into autonomous agents that must recognize uncertainty to decide when to act or seek help, though it is incremental in building on existing knowledge about confidence extraction.
The study tackled the problem of whether large language models actively use internal confidence signals to regulate behavior, finding through a four-phase abstention paradigm that confidence is the dominant predictor of abstention decisions, with effect sizes an order of magnitude larger than other factors, and causal evidence from activation steering showed manipulating confidence shifted abstention rates.
Metacognition -- the ability to assess one's own cognitive performance -- is documented across species, with internal confidence estimates serving as a key signal for adaptive behavior. While confidence can be extracted from Large Language Model (LLM) outputs, whether models actively use these signals to regulate behavior remains a fundamental question. We investigate this through a four-phase abstention paradigm.Phase 1 established internal confidence estimates in the absence of an abstention option. Phase 2 revealed that LLMs apply implicit thresholds to these estimates when deciding to answer or abstain. Confidence emerged as the dominant predictor of behavior, with effect sizes an order of magnitude larger than knowledge retrieval accessibility (RAG scores) or surface-level semantic features. Phase 3 provided causal evidence through activation steering: manipulating internal confidence signals correspondingly shifted abstention rates. Finally, Phase 4 demonstrated that models can systematically vary abstention policies based on instructed thresholds.Our findings indicate that abstention arises from the joint operation of internal confidence representations and threshold-based policies, mirroring the two-stage metacognitive control found in biological systems. This capacity is essential as LLMs transition into autonomous agents that must recognize their own uncertainty to decide when to act or seek help.