Discover-then-Name: Task-Agnostic Concept Bottlenecks via Automated Concept Discovery
This addresses the problem of interpretability in deep learning for researchers and practitioners by offering a more reliable method for building CBMs, though it is incremental as it builds on existing CBM and mechanistic interpretability techniques.
The paper tackles the challenge of ensuring that concept bottleneck models (CBMs) use detectable concepts by proposing a task-agnostic approach that first discovers concepts learned by the model using sparse autoencoders and then names them for classification, resulting in semantically meaningful concepts and performant CBMs across multiple datasets and CLIP architectures.
Concept Bottleneck Models (CBMs) have recently been proposed to address the 'black-box' problem of deep neural networks, by first mapping images to a human-understandable concept space and then linearly combining concepts for classification. Such models typically require first coming up with a set of concepts relevant to the task and then aligning the representations of a feature extractor to map to these concepts. However, even with powerful foundational feature extractors like CLIP, there are no guarantees that the specified concepts are detectable. In this work, we leverage recent advances in mechanistic interpretability and propose a novel CBM approach -- called Discover-then-Name-CBM (DN-CBM) -- that inverts the typical paradigm: instead of pre-selecting concepts based on the downstream classification task, we use sparse autoencoders to first discover concepts learnt by the model, and then name them and train linear probes for classification. Our concept extraction strategy is efficient, since it is agnostic to the downstream task, and uses concepts already known to the model. We perform a comprehensive evaluation across multiple datasets and CLIP architectures and show that our method yields semantically meaningful concepts, assigns appropriate names to them that make them easy to interpret, and yields performant and interpretable CBMs. Code available at https://github.com/neuroexplicit-saar/discover-then-name.