Contextual Prediction Difference Analysis for Explaining Individual Image Classifications
This work addresses the need for model-agnostic explanations in image classification, particularly for saturated classifiers, but it is incremental as it builds on existing PDA methods.
The authors tackled the problem of explaining individual image classifications by deep neural networks, showing that existing Prediction Difference Analysis (PDA) can fail with saturated classifiers and proposing Contextual PDA, which runs hundreds of times faster and demonstrates superiority in experiments on state-of-the-art networks.
Much effort has been devoted to understanding the decisions of deep neural networks in recent years. A number of model-aware saliency methods were proposed to explain individual classification decisions by creating saliency maps. However, they are not applicable when the parameters and the gradients of the underlying models are unavailable. Recently, model-agnostic methods have also received attention. As one of them, \textit{Prediction Difference Analysis} (PDA), a probabilistic sound methodology, was proposed. In this work, we first show that PDA can suffer from saturated classifiers. The saturation phenomenon of classifiers exists widely in current neural network-based classifiers. To explain the decisions of saturated classifiers better, we further propose Contextual PDA, which runs hundreds of times faster than PDA. The experiments show the superiority of our method by explaining image classifications of the state-of-the-art deep convolutional neural networks.