EMaP: Explainable AI with Manifold-based Perturbations
This work addresses the need for more reliable explainable AI methods, particularly for users relying on black-box models, though it is incremental as it builds on existing perturbation-based approaches.
The paper tackles the problem of generating faithful and robust explanations for black-box AI models by introducing a novel perturbation scheme that perturbs along orthogonal directions of the input manifold, showing improvements in preserving data topology and overcoming attacks against perturbation-based methods.
In the last few years, many explanation methods based on the perturbations of input data have been introduced to improve our understanding of decisions made by black-box models. The goal of this work is to introduce a novel perturbation scheme so that more faithful and robust explanations can be obtained. Our study focuses on the impact of perturbing directions on the data topology. We show that perturbing along the orthogonal directions of the input manifold better preserves the data topology, both in the worst-case analysis of the discrete Gromov-Hausdorff distance and in the average-case analysis via persistent homology. From those results, we introduce EMaP algorithm, realizing the orthogonal perturbation scheme. Our experiments show that EMaP not only improves the explainers' performance but also helps them overcome a recently-developed attack against perturbation-based methods.