Maximally Invariant Data Perturbation as Explanation
This provides a mathematically grounded explanation method for ML practitioners, though it is incremental as it adapts existing adversarial concepts to a new context.
The paper tackles the lack of formal definitions in feature scoring methods for explaining complex ML models by proposing a novel definition using maximally invariant data perturbation, inspired by adversarial examples, and shows it effectively identifies relevant image parts in VGG16 classification experiments.
While several feature scoring methods are proposed to explain the output of complex machine learning models, most of them lack formal mathematical definitions. In this study, we propose a novel definition of the feature score using the maximally invariant data perturbation, which is inspired from the idea of adversarial example. In adversarial example, one seeks the smallest data perturbation that changes the model's output. In our proposed approach, we consider the opposite: we seek the maximally invariant data perturbation that does not change the model's output. In this way, we can identify important input features as the ones with small allowable data perturbations. To find the maximally invariant data perturbation, we formulate the problem as linear programming. The experiment on the image classification with VGG16 shows that the proposed method could identify relevant parts of the images effectively.