How explainable AI affects human performance: A systematic review of the behavioural consequences of saliency maps
This review addresses the practical utility of explainable AI for human users, highlighting incremental insights by synthesizing existing behavioral studies to guide future research design.
The systematic review of 68 user studies examined whether saliency maps improve human performance in AI-assisted tasks, finding that benefits are inconsistent, with null effects or costs common, and effects vary based on task type and AI prediction correctness.
Saliency maps can explain how deep neural networks classify images. But are they actually useful for humans? The present systematic review of 68 user studies found that while saliency maps can enhance human performance, null effects or even costs are quite common. To investigate what modulates these effects, the empirical outcomes were organised along several factors related to the human tasks, AI performance, XAI methods, images to be classified, human participants and comparison conditions. In image-focused tasks, benefits were less common than in AI-focused tasks, but the effects depended on the specific cognitive requirements. Moreover, benefits were usually restricted to incorrect AI predictions in AI-focused tasks but to correct ones in image-focused tasks. XAI-related factors had surprisingly little impact. The evidence was limited for image- and human-related factors and the effects were highly dependent on the comparison conditions. These findings may support the design of future user studies.