Improving Steering Vectors by Targeting Sparse Autoencoder Features
This work addresses the challenge of predictable and robust model control for AI practitioners, though it is incremental as it builds on prior steering and SAE techniques.
The paper tackles the problem of controlling language model behavior via steering vectors by developing SAE-Targeted Steering (SAE-TS), which uses sparse autoencoders to measure causal effects and target specific features while minimizing side effects, showing it balances steering and coherence better than existing methods on various tasks.
To control the behavior of language models, steering methods attempt to ensure that outputs of the model satisfy specific pre-defined properties. Adding steering vectors to the model is a promising method of model control that is easier than finetuning, and may be more robust than prompting. However, it can be difficult to anticipate the effects of steering vectors produced by methods such as CAA [Panickssery et al., 2024] or the direct use of SAE latents [Templeton et al., 2024]. In our work, we address this issue by using SAEs to measure the effects of steering vectors, giving us a method that can be used to understand the causal effect of any steering vector intervention. We use this method for measuring causal effects to develop an improved steering method, SAE-Targeted Steering (SAE-TS), which finds steering vectors to target specific SAE features while minimizing unintended side effects. We show that overall, SAE-TS balances steering effects with coherence better than CAA and SAE feature steering, when evaluated on a range of tasks.