Interpreting Undesirable Pixels for Image Classification on Black-Box Models
This work addresses the need for better interpretability in black-box models for researchers and practitioners, but it is incremental as it builds on existing explanation methods by focusing on undesirable regions.
The paper tackles the problem of interpreting black-box image classification models by proposing a method to visualize undesirable pixels that interfere with predictions, dividing them into factors for target and non-target classes, and provides qualitative heatmaps and quantitative evaluation on ImageNet.
In an effort to interpret black-box models, researches for developing explanation methods have proceeded in recent years. Most studies have tried to identify input pixels that are crucial to the prediction of a classifier. While this approach is meaningful to analyse the characteristic of blackbox models, it is also important to investigate pixels that interfere with the prediction. To tackle this issue, in this paper, we propose an explanation method that visualizes undesirable regions to classify an image as a target class. To be specific, we divide the concept of undesirable regions into two terms: (1) factors for a target class, which hinder that black-box models identify intrinsic characteristics of a target class and (2) factors for non-target classes that are important regions for an image to be classified as other classes. We visualize such undesirable regions on heatmaps to qualitatively validate the proposed method. Furthermore, we present an evaluation metric to provide quantitative results on ImageNet.