Evaluating Sparse Autoencoders on Targeted Concept Erasure Tasks
This addresses the bottleneck of lacking high-quality performance metrics for SAE development in interpretability research, though it is incremental as it builds on prior work like SHIFT.
The paper tackled the problem of evaluating Sparse Autoencoders (SAEs) by introducing automated metrics based on SHIFT and Targeted Probe Perturbation (TPP), which effectively differentiate between SAE training hyperparameters and architectures across multiple models.
Sparse Autoencoders (SAEs) are an interpretability technique aimed at decomposing neural network activations into interpretable units. However, a major bottleneck for SAE development has been the lack of high-quality performance metrics, with prior work largely relying on unsupervised proxies. In this work, we introduce a family of evaluations based on SHIFT, a downstream task from Marks et al. (Sparse Feature Circuits, 2024) in which spurious cues are removed from a classifier by ablating SAE features judged to be task-irrelevant by a human annotator. We adapt SHIFT into an automated metric of SAE quality; this involves replacing the human annotator with an LLM. Additionally, we introduce the Targeted Probe Perturbation (TPP) metric that quantifies an SAE's ability to disentangle similar concepts, effectively scaling SHIFT to a wider range of datasets. We apply both SHIFT and TPP to multiple open-source models, demonstrating that these metrics effectively differentiate between various SAE training hyperparameters and architectures.