Interpretability as Compression: Reconsidering SAE Explanations of Neural Activations with MDL-SAEs
This work addresses interpretability challenges in neural networks for researchers by offering a more principled approach to feature extraction, though it is incremental as it builds on existing SAE methods.
The authors tackled the problem of naive Sparse Autoencoder (SAE) optimization leading to overly wide and sparse features by proposing an information-theoretic framework based on the Minimal Description Length (MDL) principle, which resulted in SAEs that prioritize features like significant line segments over memorized digits or fragments, as demonstrated on MNIST.
Sparse Autoencoders (SAEs) have emerged as a useful tool for interpreting the internal representations of neural networks. However, naively optimising SAEs for reconstruction loss and sparsity results in a preference for SAEs that are extremely wide and sparse. We present an information-theoretic framework for interpreting SAEs as lossy compression algorithms for communicating explanations of neural activations. We appeal to the Minimal Description Length (MDL) principle to motivate explanations of activations which are both accurate and concise. We further argue that interpretable SAEs require an additional property, "independent additivity": features should be able to be understood separately. We demonstrate an example of applying our MDL-inspired framework by training SAEs on MNIST handwritten digits and find that SAE features representing significant line segments are optimal, as opposed to SAEs with features for memorised digits from the dataset or small digit fragments. We argue that using MDL rather than sparsity may avoid potential pitfalls with naively maximising sparsity such as undesirable feature splitting and that this framework naturally suggests new hierarchical SAE architectures which provide more concise explanations.