Mitigating Overthinking in Large Reasoning Language Models via Reasoning Path Deviation Monitoring
This addresses efficiency and accuracy issues for users of large reasoning models, but it is incremental as it builds on early-exit strategies.
The paper tackles the problem of overthinking in Large Reasoning Language Models, which degrades performance and efficiency, by proposing an early-exit method that monitors reasoning path deviation to dynamically terminate redundant steps, resulting in the largest performance improvement over vanilla Chain-of-Thought compared to existing methods.
Large Reasoning Language Models (LRLMs) demonstrate impressive capabilities on complex tasks by utilizing long Chain-of-Thought reasoning. However, they are prone to overthinking, which generates redundant reasoning steps that degrade both performance and efficiency. Recently, early-exit strategies are proposed to mitigate overthinking by dynamically and adaptively terminating redundant reasoning. However, current early-exit methods either introduce extra training overhead by relying on proxy models or limit inference throughput due to the frequent content switching between reasoning and generating probing answers. Moreover, most early-exit methods harm LRLMs performance due to over-truncation. Our insight stems from an observation: overthinking often causes LRLMs to deviate from the correct reasoning path, which is frequently accompanied by high-entropy transition tokens. Given this, we propose an early-exit method deeply coupled with the native reasoning process, which leverages the path deviation index as a dedicated monitoring metric for the frequent occurrence of high-entropy transition tokens to dynamically detect and terminate overthinking trajectories. We conduct experiments across multiple benchmarks using LRLMs of different types and scales, and the results indicate that our method delivers the largest performance improvement over vanilla CoT compared to existing early-exit methods.