Interpreting Language Models Through Concept Descriptions: A Survey
This work addresses the problem of making neural network decision-making more transparent for researchers and practitioners in AI interpretability, but it is incremental as it synthesizes existing research rather than introducing new methods.
This survey paper tackles the challenge of interpreting Large Language Models (LLMs) by reviewing methods that generate natural language concept descriptions for model components, such as neurons and attention heads, and identifies a need for more rigorous, causal evaluation in the field.
Understanding the decision-making processes of neural networks is a central goal of mechanistic interpretability. In the context of Large Language Models (LLMs), this involves uncovering the underlying mechanisms and identifying the roles of individual model components such as neurons and attention heads, as well as model abstractions such as the learned sparse features extracted by Sparse Autoencoders (SAEs). A rapidly growing line of work tackles this challenge by using powerful generator models to produce open-vocabulary, natural language concept descriptions for these components. In this paper, we provide the first survey of the emerging field of concept descriptions for model components and abstractions. We chart the key methods for generating these descriptions, the evolving landscape of automated and human metrics for evaluating them, and the datasets that underpin this research. Our synthesis reveals a growing demand for more rigorous, causal evaluation. By outlining the state of the art and identifying key challenges, this survey provides a roadmap for future research toward making models more transparent.