ThinkPilot: Steering Reasoning Models via Automated Think-prefixes Optimization
It addresses reasoning inefficiencies in AI models for developers and researchers, offering a novel optimization method that is incremental in combining evolutionary processes with existing training-based approaches.
The paper tackles the problem of inefficient and off-target reasoning in Large Reasoning Models by introducing ThinkPilot, a training-free framework that automatically optimizes reasoning through evolutionary think-prefixes, resulting in significant improvements such as cutting the StrongREJECT score from 27.0% to 0.7% and enhancing accuracy-length trade-offs.
Large Reasoning Models (LRMs) are powerful, but they still suffer from inefficient and off-target reasoning. Currently, training-free methods are limited to either rigid heuristics or descriptive, non-actionable analyses. In this paper, we introduce ThinkPilot, a training-free framework that automatically optimizes LRMs reasoning. It uses an evolutionary process to generate think-prefixes, which are instructions that evolve driven by a taxonomy of reasoning behaviors to guide models toward superior performance. Extensive experiments demonstrate ThinkPilot's broad effectiveness: it significantly improves the accuracy-length trade-off for efficient reasoning, drastically improves safety (for example, cutting the StrongREJECT score of DeepSeek-R1-Distill-Qwen-32B from 27.0% to 0.7), and enhances instruction following. It also synergizes with existing training-based methods. Our analysis reveals that think-prefixes can reliably control LRMs' reasoning behaviors, and that different tasks have strong preferences for specific behavioral distributions. By automatically identifying and eliciting these behaviors, ThinkPilot provides a generalizable framework for aligning LRMs reasoning with task demands. Data and code are available at https://github.com/teqkilla/ThinkPilot