STU-PID: Steering Token Usage via PID Controller for Efficient Large Language Model Reasoning
This addresses computational inefficiency in LLM reasoning for AI practitioners, offering a dynamic alternative to static methods.
The paper tackles the overthinking problem in large language models during chain-of-thought reasoning, which leads to excessive computational costs and potential performance degradation. It introduces STUPID, a training-free method using a PID controller to dynamically adjust activation steering, resulting in a 6% accuracy improvement and 32% token reduction on GSM8K.
Large Language Models employing extended chain-of-thought (CoT) reasoning often suffer from the overthinking phenomenon, generating excessive and redundant reasoning steps that increase computational costs while potentially degrading performance. While recent work has explored static steering approaches to mitigate this issue, they lack the adaptability to dynamically adjust intervention strength based on real-time reasoning quality. We propose STUPID (Steering Token Usage via PID controller), a novel training-free method that employs a PID controller to dynamically modulate activation steering strength during inference. Our approach combines a chunk-level classifier for detecting redundant reasoning patterns with a PID control mechanism that adaptively adjusts steering intensity based on the predicted redundancy probability. Experimental evaluation on GSM8K demonstrates that STUPID achieves a 6% improvement in accuracy while reducing token usage by 32%, outperforming static steering baselines. Our method provides a principled framework for dynamic reasoning calibration that maintains reasoning quality while significantly improving computational efficiency.