Think Twice Before You Write -- an Entropy-based Decoding Strategy to Enhance LLM Reasoning
This work addresses a bottleneck in LLM reasoning for AI researchers and practitioners, offering an incremental improvement over existing methods like self-consistency.
The paper tackles the problem of error propagation and computational inefficiency in decoding strategies for large language models by proposing an entropy-guided decoding framework that adaptively branches on high-uncertainty tokens, achieving strong accuracy on benchmarks like GSM8K and AMC2023 while reducing costs compared to larger models.
Decoding strategies play a central role in shaping the reasoning ability of large language models (LLMs). Traditional methods such as greedy decoding and beam search often suffer from error propagation, while sampling-based approaches introduce randomness without adequate robustness. Self-consistency improves reliability by aggregating multiple rollouts, but incurs significant computational overhead. We propose an entropy-guided decoding framework that introduces token-level adaptivity into generation. At each step, the model computes the entropy of the token distribution, identifies high-uncertainty positions, and selectively branches on these vulnerable points. A dynamic pool of partial rollouts is maintained and expanded until solutions are completed, concentrating computation where uncertainty is greatest and avoiding unnecessary exploration in confident regions. To enable efficient termination, we apply a rollout-level Entropy After </Think> (EAT) stopping criterion by performing entropy evaluation after the full reasoning trace, rather than incrementally at every step. Experiments on GSM8K, AMC2023, and their perturbed variants demonstrate that our method achieves consistently strong accuracy. Notably, on smaller LLMs, performance is comparable to GPT-5 while operating at a fraction of the cost.