A Unifying Framework for Unsupervised Concept Extraction
For researchers in interpretability and mechanistic interpretability, this work provides a foundational theory to understand and improve concept extraction methods.
The paper presents a unified theoretical framework for unsupervised concept extraction, framing it as identifying a generative model, and provides a meta-theorem that simplifies proving identifiability guarantees for existing approaches.
Techniques for concept extraction, such as sparse autoencoders and transcoders, aim to extract high-level symbolic concepts from low-level nonsymbolic representations. When these extracted concepts are used for downstream tasks such as model steering and unlearning, it is essential to understand their guarantees, or lack thereof. In this work, we present a unified theoretical framework for unsupervised concept extraction, in which we frame the task of concept extraction as identifying a generative model. We present a general meta-theorem for identifiability, which reduces the problem of establishing identifiability guarantees to the problem of characterizing the intersection of two sets. As we demonstrate on a range of widely-used approaches, this meta-theorem substantially simplifies the task of proving such guarantees, thus paving the way for the development of new, principled approaches for concept extraction.