Prototype-Based Dynamic Steering for Large Language Models
This addresses the need for adaptive, instruction-free reasoning enhancement in LLMs, offering a lightweight alternative to training-based methods, though it is incremental in building on existing steering techniques.
The paper tackles the problem of amplifying large language model reasoning without explicit instructions or fine-tuning by introducing Prototype-Based Dynamic Steering, which improves accuracy on tasks like GSM8H, AQuA-RAT, and BIG-Bench, with gains persisting even when Chain-of-Thought is suppressed.
Despite impressive breadth, LLMs still rely on explicit reasoning instructions or static, one-fits-all steering methods, leaving a gap for adaptive, instruction-free reasoning amplification. We present Prototype-Based Dynamic Steering (PDS), a test-time method that amplifies large language model (LLM) reasoning without adding or altering instructions. We introduce "reasoning prototypes" by clustering activation differences between Chain-of-Thought (CoT) and neutral prompts. At inference, an input's hidden state is projected onto these prototypes to form an instance-specific steering vector. Evaluated on GSM8K, AQuA-RAT, and BIG-Bench tasks, PDS consistently improves accuracy without fine-tuning or prompt engineering. Notably, the gains persist even when CoT is explicitly suppressed to improve cost-efficiency, indicating that the intervention strengthens latent reasoning processes rather than inducing a superficial behavioral shift. These results position dynamic, prototype-guided steering as a lightweight alternative to training-time approaches for enhancing LLM reasoning.