Certainty-Guided Reasoning in Large Language Models: A Dynamic Thinking Budget Approach
This work addresses efficiency and reliability issues for users of large reasoning language models in domains like exams, making it incremental by adapting existing generator/discriminator ideas to reasoning.
The paper tackles the problem of inefficient and unreliable reasoning in large language models by proposing Certainty-Guided Reasoning (CGR), which dynamically adjusts reasoning tokens based on confidence thresholds, resulting in improved accuracy on AIME2024/2025 datasets and reduced token usage with tunable trade-offs.
The rise of large reasoning language models (LRLMs) has unlocked new potential for solving complex tasks. These models operate with a thinking budget, that is, a predefined number of reasoning tokens used to arrive at a solution. We propose a novel approach, inspired by the generator/discriminator framework in generative adversarial networks, in which a critic model periodically probes its own reasoning to assess whether it has reached a confident conclusion. If not, reasoning continues until a target certainty threshold is met. This mechanism adaptively balances efficiency and reliability by allowing early termination when confidence is high, while encouraging further reasoning when uncertainty persists. Through experiments on the AIME2024 and AIME2025 datasets, we show that Certainty-Guided Reasoning (CGR) improves baseline accuracy while reducing token usage. Importantly, extended multi-seed evaluations over 64 runs demonstrate that CGR is stable, reducing variance across seeds and improving exam-like performance under penalty-based grading. Additionally, our token savings analysis shows that CGR can eliminate millions of tokens in aggregate, with tunable trade-offs between certainty thresholds and efficiency. Together, these findings highlight certainty as a powerful signal for reasoning sufficiency. By integrating confidence into the reasoning process, CGR makes large reasoning language models more adaptive, trustworthy, and resource efficient, paving the way for practical deployment in domains where both accuracy and computational cost matter.