Right for the Right Reason: Making Image Classification Robust
This addresses the issue of interpretability and robustness in image classification for AI practitioners, though it is incremental as it builds on existing explanation and object detection methods.
The paper tackles the problem of image classification models making correct predictions for the wrong reasons, such as relying on incidental evidence, by proposing a new explanation quality metric called ObAlEx to measure object-aligned explanations. They show that additional training can improve model focus on actual evidence without reducing accuracy.
The effectiveness of Convolutional Neural Networks (CNNs)in classifying image data has been thoroughly demonstrated. In order to explain the classification to humans, methods for visualizing classification evidence have been developed in recent years. These explanations reveal that sometimes images are classified correctly, but for the wrong reasons,i.e., based on incidental evidence. Of course, it is desirable that images are classified correctly for the right reasons, i.e., based on the actual evidence. To this end, we propose a new explanation quality metric to measure object aligned explanation in image classification which we refer to as theObAlExmetric. Using object detection approaches, explanation approaches, and ObAlEx, we quantify the focus of CNNs on the actual evidence. Moreover, we show that additional training of the CNNs can improve the focus of CNNs without decreasing their accuracy.