Leakage and Interpretability in Concept-Based Models
This addresses leakage issues in interpretable AI models for high-risk deployment scenarios, providing a rigorous analysis and practical guidelines, though it is incremental in refining existing methods.
The paper tackles the problem of information leakage in Concept Bottleneck Models, which compromises interpretability, and introduces an information-theoretic framework with CTL and ICL scores to quantify leakage, showing these measures predict model behavior and outperform existing alternatives.
Concept Bottleneck Models aim to improve interpretability by predicting high-level intermediate concepts, representing a promising approach for deployment in high-risk scenarios. However, they are known to suffer from information leakage, whereby models exploit unintended information encoded within the learned concepts. We introduce an information-theoretic framework to rigorously characterise and quantify leakage, and define two complementary measures: the concepts-task leakage (CTL) and interconcept leakage (ICL) scores. We show that these measures are strongly predictive of model behaviour under interventions and outperform existing alternatives in robustness and reliability. Using this framework, we identify the primary causes of leakage and provide strong evidence that Concept Embedding Models exhibit substantial leakage regardless of the hyperparameters choice. Finally, we propose practical guidelines for designing concept-based models to reduce leakage and ensure interpretability.