Identifying Intervenable and Interpretable Features via Orthogonality Regularization
This work addresses interpretability and causal intervention in language models, offering a method to enhance modular representations, though it is incremental as it builds on existing sparse autoencoder techniques.
The paper tackles the problem of feature interference and superposition in language model fine-tuning by introducing an orthogonality penalty to disentangle decoder matrix features, resulting in identifiable and interpretable features with minimal performance loss on the target dataset.
With recent progress on fine-tuning language models around a fixed sparse autoencoder, we disentangle the decoder matrix into almost orthogonal features. This reduces interference and superposition between the features, while keeping performance on the target dataset essentially unchanged. Our orthogonality penalty leads to identifiable features, ensuring the uniqueness of the decomposition. Further, we find that the distance between embedded feature explanations increases with stricter orthogonality penalty, a desirable property for interpretability. Invoking the $\textit{Independent Causal Mechanisms}$ principle, we argue that orthogonality promotes modular representations amenable to causal intervention. We empirically show that these increasingly orthogonalized features allow for isolated interventions. Our code is available under $\texttt{https://github.com/mrtzmllr/sae-icm}$.