Evaluation of Saliency-based Explainability Method
This work addresses the need for better human-centered evaluation of explainability methods for users of AI systems, but it is incremental as it focuses on assessing existing methods rather than introducing new ones.
The study tackled the lack of rigorous evaluation for saliency-based explainability methods in AI by conducting three human subject experiments to gauge their effectiveness, finding that current evidence is largely anecdotal.
A particular class of Explainable AI (XAI) methods provide saliency maps to highlight part of the image a Convolutional Neural Network (CNN) model looks at to classify the image as a way to explain its working. These methods provide an intuitive way for users to understand predictions made by CNNs. Other than quantitative computational tests, the vast majority of evidence to highlight that the methods are valuable is anecdotal. Given that humans would be the end-users of such methods, we devise three human subject experiments through which we gauge the effectiveness of these saliency-based explainability methods.