Can we Constrain Concept Bottleneck Models to Learn Semantically Meaningful Input Features?
This addresses the issue of ensuring trust in interpretable AI models for users by improving feature attribution, though it is incremental as it focuses on dataset conditions rather than a new method.
The paper tackles the problem that Concept Bottleneck Models (CBMs) often rely on irrelevant input features for concept predictions, which undermines their interpretability. It demonstrates that by using datasets with clear links between input features and concepts, such as avoiding co-occurring concepts, CBMs can learn to attribute semantically meaningful features to correct predictions, validated on synthetic and real-world image datasets.
Concept Bottleneck Models (CBMs) are regarded as inherently interpretable because they first predict a set of human-defined concepts which are used to predict a task label. For inherent interpretability to be fully realised, and ensure trust in a model's output, it's desirable for concept predictions to use semantically meaningful input features. For instance, in an image, pixels representing a broken bone should contribute to predicting a fracture. However, current literature suggests that concept predictions often rely on irrelevant input features. We hypothesise that this occurs when dataset labels include inaccurate concept annotations, or the relationship between input features and concepts is unclear. In general, the effect of dataset labelling on concept representations remains an understudied area. In this paper, we demonstrate that CBMs can learn to map concepts to semantically meaningful input features, by utilising datasets with a clear link between the input features and the desired concept predictions. This is achieved, for instance, by ensuring multiple concepts do not always co-occur and, therefore provide a clear training signal for the CBM to distinguish the relevant input features for each concept. We validate our hypothesis on both synthetic and real-world image datasets, and demonstrate under the correct conditions, CBMs can learn to attribute semantically meaningful input features to the correct concept predictions.