Between Underthinking and Overthinking: An Empirical Study of Reasoning Length and correctness in LLMs
This work addresses the problem of inefficient reasoning in LLMs for AI researchers, though it is incremental as it builds on existing evidence about reasoning length effects.
The study investigates the relationship between reasoning length and answer correctness in LLMs, finding that models overthink simple problems and underthink harder ones, leading to misjudged difficulty and inappropriate response lengths. Experiments show that length reduction via preference optimization can significantly shorten outputs while maintaining acceptable accuracy.
Large language models (LLMs) are increasingly optimized for long reasoning, under the assumption that more reasoning leads to better performance. However, emerging evidence suggests that longer responses can sometimes degrade accuracy rather than improve it. In this paper, we conduct a systematic empirical study of the relationship between reasoning length and answer correctness. We find that LLMs tend to overthink simple problems, generating unnecessarily long outputs, and underthink harder ones, failing to extend their reasoning when it is most needed. This indicates that models might misjudge problem difficulty and fail to calibrate their response length appropriately. Furthermore, we investigate the effects of length reduction with a preference optimization algorithm when simply preferring the shorter responses regardless of answer correctness. Experiments show that the generation length can be significantly reduced while maintaining acceptable accuracy. Our findings highlight generation length as a meaningful signal for reasoning behavior and motivate further exploration into LLMs' self-awareness in reasoning length adaptation.