EMAP: Explanation by Minimal Adversarial Perturbation
This work addresses the need for faster and more interpretable model explanations in data-heavy industries, though it is incremental as it builds on existing explanation paradigms.
The paper tackles the problem of explaining black-box classifier decisions by introducing EMAP, which finds the minimal adversarial perturbation needed to cause misclassification, combining feature-weighting and counterfactual paradigms. The result is a method that provides implicit confidence estimates and improves speed by an order of magnitude over sampling-based approaches like LIME.
Modern instance-based model-agnostic explanation methods (LIME, SHAP, L2X) are of great use in data-heavy industries for model diagnostics, and for end-user explanations. These methods generally return either a weighting or subset of input features as an explanation of the classification of an instance. An alternative literature argues instead that counterfactual instances provide a more useable characterisation of a black box classifier's decisions. We present EMAP, a neural network based approach which returns as Explanation the Minimal Adversarial Perturbation to an instance required to cause the underlying black box model to missclassify. We show that this approach combines the two paradigms, recovering the output of feature-weighting methods in continuous feature spaces, whilst also indicating the direction in which the nearest counterfactuals can be found. Our method also provides an implicit confidence estimate in its own explanations, adding a clarity to model diagnostics other methods lack. Additionally, EMAP improves upon the speed of sampling-based methods such as LIME by an order of magnitude, allowing for model explanations in time-critical applications, or at the dataset level, where sampling-based methods are infeasible. We extend our approach to categorical features using a partitioned Gumbel layer, and demonstrate its efficacy on several standard datasets.