Regularizing Black-box Models for Improved Interpretability (HILL 2019 Version)
This work addresses the problem of achieving interpretable machine learning models without sacrificing accuracy, which is crucial for users in fields requiring transparent AI decisions, though it is incremental as it builds on existing regularization and interpretability techniques.
The paper tackles the trade-off between accuracy and interpretability in machine learning by introducing a method to regularize black-box models for improved interpretability during training, resulting in substantial improvements in explanation fidelity and stability across various datasets and explanation systems while slightly increasing accuracy.
Most of the work on interpretable machine learning has focused on designing either inherently interpretable models, which typically trade-off accuracy for interpretability, or post-hoc explanation systems, which lack guarantees about their explanation quality. We propose an alternative to these approaches by directly regularizing a black-box model for interpretability at training time. Our approach explicitly connects three key aspects of interpretable machine learning: (i) the model's innate explainability, (ii) the explanation system used at test time, and (iii) the metrics that measure explanation quality. Our regularization results in substantial improvement in terms of the explanation fidelity and stability metrics across a range of datasets and black-box explanation systems while slightly improving accuracy. Further, if the resulting model is still not sufficiently interpretable, the weight of the regularization term can be adjusted to achieve the desired trade-off between accuracy and interpretability. Finally, we justify theoretically that the benefits of explanation-based regularization generalize to unseen points.