Sparse Linear Concept Discovery Models
This addresses the problem of creating more interpretable and accurate models for safety-critical applications, though it is incremental as it builds on existing CBM approaches.
The paper tackles the performance degradation and low interpretability of Concept Bottleneck Models (CBMs) by proposing a framework using Contrastive Language Image models and a sparse linear layer, which outperforms recent CBM approaches in accuracy and yields high concept sparsity for easier investigation.
The recent mass adoption of DNNs, even in safety-critical scenarios, has shifted the focus of the research community towards the creation of inherently intrepretable models. Concept Bottleneck Models (CBMs) constitute a popular approach where hidden layers are tied to human understandable concepts allowing for investigation and correction of the network's decisions. However, CBMs usually suffer from: (i) performance degradation and (ii) lower interpretability than intended due to the sheer amount of concepts contributing to each decision. In this work, we propose a simple yet highly intuitive interpretable framework based on Contrastive Language Image models and a single sparse linear layer. In stark contrast to related approaches, the sparsity in our framework is achieved via principled Bayesian arguments by inferring concept presence via a data-driven Bernoulli distribution. As we experimentally show, our framework not only outperforms recent CBM approaches accuracy-wise, but it also yields high per example concept sparsity, facilitating the individual investigation of the emerging concepts.