RISER: Orchestrating Latent Reasoning Skills for Adaptive Activation Steering
This addresses the need for more controllable and efficient reasoning in LLMs, offering a parameter-efficient alternative to training-intensive methods, though it is incremental as it builds on existing activation steering techniques.
The paper tackled the problem of static activation steering in large language models by proposing RISER, a plug-and-play framework that adaptively composes reusable reasoning vectors, resulting in 3.4-6.5% average zero-shot accuracy improvements and 2-3x higher token efficiency compared to chain-of-thought reasoning.
Recent work on domain-specific reasoning with large language models (LLMs) often relies on training-intensive approaches that require parameter updates. While activation steering has emerged as a parameter efficient alternative, existing methods apply static, manual interventions that fail to adapt to the dynamic nature of complex reasoning. To address this limitation, we propose RISER (Router-based Intervention for Steerable Enhancement of Reasoning), a plug-and-play intervention framework that adaptively steers LLM reasoning in activation space. RISER constructs a library of reusable reasoning vectors and employs a lightweight Router to dynamically compose them for each input. The Router is optimized via reinforcement learning under task-level rewards, activating latent cognitive primitives in an emergent and compositional manner. Across seven diverse benchmarks, RISER yields 3.4-6.5% average zero-shot accuracy improvements over the base model while surpassing CoT-style reasoning with 2-3x higher token efficiency and robust accuracy gains. Further analysis shows that RISER autonomously combines multiple vectors into interpretable, precise control strategies, pointing toward more controllable and efficient LLM reasoning.