LIMIS: Locally Interpretable Modeling using Instance-wise Subsampling
This addresses the need for high-fidelity local explanations in machine learning, which is crucial for trust and adoption, though it is an incremental improvement over existing locally interpretable modeling methods.
The paper tackles the problem of low fidelity in locally interpretable models for explaining black-box predictions by proposing LIMIS, a framework that uses instance-wise subsampling and policy gradients to train low-capacity interpretable models, achieving near-matching accuracy to black-box models and outperforming state-of-the-art methods on multiple tabular datasets.
Understanding black-box machine learning models is crucial for their widespread adoption. Learning globally interpretable models is one approach, but achieving high performance with them is challenging. An alternative approach is to explain individual predictions using locally interpretable models. For locally interpretable modeling, various methods have been proposed and indeed commonly used, but they suffer from low fidelity, i.e. their explanations do not approximate the predictions well. In this paper, our goal is to push the state-of-the-art in high-fidelity locally interpretable modeling. We propose a novel framework, Locally Interpretable Modeling using Instance-wise Subsampling (LIMIS). LIMIS utilizes a policy gradient to select a small number of instances and distills the black-box model into a low-capacity locally interpretable model using those selected instances. Training is guided with a reward obtained directly by measuring the fidelity of the locally interpretable models. We show on multiple tabular datasets that LIMIS near-matches the prediction accuracy of black-box models, significantly outperforming state-of-the-art locally interpretable models in terms of fidelity and prediction accuracy.