Compute Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders
This addresses a bottleneck in interpretability research for neural networks, particularly large language models, by improving sparse inference in SAEs, though it appears incremental as it builds on existing SAE and compressed sensing theory.
The paper tackles the problem of sparse autoencoders (SAEs) being insufficient for accurate sparse inference due to their simple linear-nonlinear encoding mechanism, and shows that decoupling encoding and decoding with more sophisticated methods yields substantial performance gains with minimal compute increases, generalizing to large language models for improved interpretability.
A recent line of work has shown promise in using sparse autoencoders (SAEs) to uncover interpretable features in neural network representations. However, the simple linear-nonlinear encoding mechanism in SAEs limits their ability to perform accurate sparse inference. Using compressed sensing theory, we prove that an SAE encoder is inherently insufficient for accurate sparse inference, even in solvable cases. We then decouple encoding and decoding processes to empirically explore conditions where more sophisticated sparse inference methods outperform traditional SAE encoders. Our results reveal substantial performance gains with minimal compute increases in correct inference of sparse codes. We demonstrate this generalises to SAEs applied to large language models, where more expressive encoders achieve greater interpretability. This work opens new avenues for understanding neural network representations and analysing large language model activations.