Do Sparse Autoencoders Generalize? A Case Study of Answerability
This work addresses the problem of evaluating feature generalization in interpretability methods for researchers, highlighting the need for robust evaluation approaches.
The researchers investigated whether sparse autoencoder (SAE) features for language model interpretability generalize across different answerability datasets, finding that SAE features show inconsistent out-of-domain transfer performance ranging from near-random to superior compared to residual stream probes.
Sparse autoencoders (SAEs) have emerged as a promising approach in language model interpretability, offering unsupervised extraction of sparse features. For interpretability methods to succeed, they must identify abstract features across domains, and these features can often manifest differently in each context. We examine this through "answerability" - a model's ability to recognize answerable questions. We extensively evaluate SAE feature generalization across diverse, partly self-constructed answerability datasets for Gemma 2 SAEs. Our analysis reveals that residual stream probes outperform SAE features within domains, but generalization performance differs sharply. SAE features show inconsistent out-of-domain transfer, with performance varying from almost random to outperforming residual stream probes. Overall, this demonstrates the need for robust evaluation methods and quantitative approaches to predict feature generalization in SAE-based interpretability.