A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Models
This is an incremental survey that addresses the need for interpretability in AI for researchers and practitioners working with large language models.
This paper tackles the problem of interpreting the opaque internal mechanisms of Large Language Models by surveying Sparse Autoencoders as a method to disentangle features, resulting in a comprehensive overview of their technical framework, explanation approaches, evaluation methods, and applications.
Large Language Models (LLMs) have transformed natural language processing, yet their internal mechanisms remain largely opaque. Recently, mechanistic interpretability has attracted significant attention from the research community as a means to understand the inner workings of LLMs. Among various mechanistic interpretability approaches, Sparse Autoencoders (SAEs) have emerged as a promising method due to their ability to disentangle the complex, superimposed features within LLMs into more interpretable components. This paper presents a comprehensive survey of SAEs for interpreting and understanding the internal workings of LLMs. Our major contributions include: (1) exploring the technical framework of SAEs, covering basic architecture, design improvements, and effective training strategies; (2) examining different approaches to explaining SAE features, categorized into input-based and output-based explanation methods; (3) discussing evaluation methods for assessing SAE performance, covering both structural and functional metrics; and (4) investigating real-world applications of SAEs in understanding and manipulating LLM behaviors.