Play MNIST For Me! User Studies on the Effects of Post-Hoc, Example-Based Explanations & Error Rates on Debugging a Deep Learning, Black-Box Classifier
This addresses the problem of user trust in AI systems for XAI practitioners, though it is incremental as it builds on existing explanation methods.
The paper investigated how post-hoc example-based explanations and error rates affect people's perceptions of a black-box classifier, finding that such explanations make misclassifications seem more correct and that error rates above 4% reduce trust and perceived correctness.
This paper reports two experiments (N=349) on the impact of post hoc explanations by example and error rates on peoples perceptions of a black box classifier. Both experiments show that when people are given case based explanations, from an implemented ANN CBR twin system, they perceive miss classifications to be more correct. They also show that as error rates increase above 4%, people trust the classifier less and view it as being less correct, less reasonable and less trustworthy. The implications of these results for XAI are discussed.