Measuring and improving the quality of visual explanations
This work addresses the need for standardized evaluation in explainable AI, which is crucial for safe deployment of neural networks, but it is incremental as it builds on existing methods.
The paper tackles the problem of evaluating visual explanation methods for neural networks by proposing a new evaluation procedure, and finds that combining multiple sources improves explanations while challenging the importance of bias parameters, with conclusions supported by assessments on ImageNet classifiers.
The ability of to explain neural network decisions goes hand in hand with their safe deployment. Several methods have been proposed to highlight features important for a given network decision. However, there is no consensus on how to measure effectiveness of these methods. We propose a new procedure for evaluating explanations. We use it to investigate visual explanations extracted from a range of possible sources in a neural network. We quantify the benefit of combining these sources and challenge a recent appeal for taking bias parameters into account. We support our conclusions with a general assessment of the impact of bias parameters in ImageNet classifiers