A Comparative Analysis of Sparse Autoencoder and Activation Difference in Language Model Steering
This work addresses the challenge of improving language model steering for tasks like mathematical reasoning, though it is incremental as it builds on existing SAE methods.
The paper tackled the problem of steering language models using sparse autoencoders by addressing issues with non-semantic features and degenerate outputs, resulting in SAEs outperforming mean activation difference methods on mathematical reasoning benchmarks and matching them on IF-Eval.
Sparse autoencoders (SAEs) have recently emerged as a powerful tool for language model steering. Prior work has explored top-k SAE latents for steering, but we observe that many dimensions among the top-k latents capture non-semantic features such as punctuation rather than semantic attributes like instructions. To address this, we propose focusing on a single, most relevant SAE latent (top-1), eliminating redundant features. We further identify a limitation in constant SAE steering, which often produces degenerate outputs such as repetitive single words. To mitigate this, we introduce a token-wise decaying steering strategy, enabling more faithful comparisons with mean activation difference baselines. Empirically, we show that steering an SAE latent associated with reasoning reliably elicits step-by-step mathematical reasoning and enhances inference quality, functionally resembling the effect of appending a guiding token. Our results demonstrate that SAEs outperform mean activation difference methods on mathematical reasoning benchmarks and match their performance on IF-Eval.