Learning to Receive Help: Intervention-Aware Concept Embedding Models
This addresses the issue of unreliable concept interventions in interpretable AI for users needing transparent and correctable models, representing an incremental improvement over existing CBMs.
The paper tackles the problem of concept bottleneck models (CBMs) having inconsistent receptiveness to user interventions by proposing Intervention-aware Concept Embedding models (IntCEMs), which learn an intervention policy during training to improve performance at test-time, significantly outperforming state-of-the-art models.
Concept Bottleneck Models (CBMs) tackle the opacity of neural architectures by constructing and explaining their predictions using a set of high-level concepts. A special property of these models is that they permit concept interventions, wherein users can correct mispredicted concepts and thus improve the model's performance. Recent work, however, has shown that intervention efficacy can be highly dependent on the order in which concepts are intervened on and on the model's architecture and training hyperparameters. We argue that this is rooted in a CBM's lack of train-time incentives for the model to be appropriately receptive to concept interventions. To address this, we propose Intervention-aware Concept Embedding models (IntCEMs), a novel CBM-based architecture and training paradigm that improves a model's receptiveness to test-time interventions. Our model learns a concept intervention policy in an end-to-end fashion from where it can sample meaningful intervention trajectories at train-time. This conditions IntCEMs to effectively select and receive concept interventions when deployed at test-time. Our experiments show that IntCEMs significantly outperform state-of-the-art concept-interpretable models when provided with test-time concept interventions, demonstrating the effectiveness of our approach.