Learning to Intervene on Concept Bottlenecks
This work addresses the need for more efficient and interactive interpretable AI tools for users of concept-based models, though it is incremental by building directly on existing CBM frameworks.
The paper tackles the problem of concept bottleneck models (CBMs) discarding user interventions after single use by introducing concept bottleneck memory models (CB2Ms), which store past interventions to generalize them to new situations, resulting in improved model performance with fewer required interventions, as demonstrated in experiments on distribution shifts and confounded data.
While deep learning models often lack interpretability, concept bottleneck models (CBMs) provide inherent explanations via their concept representations. Moreover, they allow users to perform interventional interactions on these concepts by updating the concept values and thus correcting the predictive output of the model. Up to this point, these interventions were typically applied to the model just once and then discarded. To rectify this, we present concept bottleneck memory models (CB2Ms), which keep a memory of past interventions. Specifically, CB2Ms leverage a two-fold memory to generalize interventions to appropriate novel situations, enabling the model to identify errors and reapply previous interventions. This way, a CB2M learns to automatically improve model performance from a few initially obtained interventions. If no prior human interventions are available, a CB2M can detect potential mistakes of the CBM bottleneck and request targeted interventions. Our experimental evaluations on challenging scenarios like handling distribution shifts and confounded data demonstrate that CB2Ms are able to successfully generalize interventions to unseen data and can indeed identify wrongly inferred concepts. Hence, CB2Ms are a valuable tool for users to provide interactive feedback on CBMs, by guiding a user's interaction and requiring fewer interventions.