Streamlining models with explanations in the learning loop
This work addresses the need for more interpretable AI models for users who rely on explanations, though it is incremental as it builds on existing explainable AI methods.
The paper tackles the problem of black-box model interpretability by integrating post-hoc explanations into the feature engineering phase, resulting in streamlined models that improve explanation compactness while maintaining accuracy.
Several explainable AI methods allow a Machine Learning user to get insights on the classification process of a black-box model in the form of local linear explanations. With such information, the user can judge which features are locally relevant for the classification outcome, and get an understanding of how the model reasons. Standard supervised learning processes are purely driven by the original features and target labels, without any feedback loop informed by the local relevance of the features identified by the post-hoc explanations. In this paper, we exploit this newly obtained information to design a feature engineering phase, where we combine explanations with feature values. To do so, we develop two different strategies, named Iterative Dataset Weighting and Targeted Replacement Values, which generate streamlined models that better mimic the explanation process presented to the user. We show how these streamlined models compare to the original black-box classifiers, in terms of accuracy and compactness of the newly produced explanations.