ARS: Adaptive Reasoning Suppression for Efficient Large Reasoning Language Models
This addresses efficiency issues for users of large reasoning models, offering a novel method to reduce inference costs without sacrificing performance, though it is incremental in improving existing suppression techniques.
The paper tackles the problem of computational inefficiency in Large Reasoning Language Models due to overthinking by proposing Adaptive Reasoning Suppression (ARS), a training-free method that dynamically suppresses redundant reasoning steps, achieving up to 57.9% reduction in energy and similar gains in tokens and latency while maintaining or improving accuracy.
Large Reasoning Language Models (LRLMs or LRMs) demonstrate remarkable capabilities in complex reasoning tasks, but suffer from significant computational inefficiencies due to overthinking phenomena. Existing efficient reasoning methods face the challenge of balancing reasoning quality with inference cost reduction. We propose \textbf{Adaptive Reasoning Suppression (ARS)}, a novel training-free approach that dynamically suppresses redundant reasoning steps while preserving accuracy through adaptive certainty monitoring. ARS introduces a multi-checkpoint certainty estimation mechanism with progressive suppression thresholds, achieving superior efficiency compared to static suppression methods. Our extensive evaluation across mathematical reasoning benchmarks using multiple model architectures demonstrates that ARS achieves up to 53%, 46.1%, and 57.9% in token, latency and energy reduction, while maintaining or improving accuracy.