The Effect of Explainable AI-based Decision Support on Human Task Performance: A Meta-Analysis
This work addresses inconsistent findings on XAI's impact on human decision-making in information systems, providing insights for the human-computer interaction field, though it is incremental as it synthesizes existing research.
The paper conducted a meta-analysis to investigate how explainable AI (XAI) affects human task performance in classification tasks, finding that XAI-based decision support improves performance, but explanations themselves are not the main driver, with risk of bias in studies moderating the effect.
The desirable properties of explanations in information systems have fueled the demands for transparency in artificial intelligence (AI) outputs. To address these demands, the field of explainable AI (XAI) has put forth methods that can support human decision-making by explaining AI outputs. However, current empirical works present inconsistent findings on whether such explanations help to improve users' task performance in decision support systems (DSS). In this paper, we conduct a meta-analysis to explore how XAI affects human performance in classification tasks. Our results show an improvement in task performance through XAI-based decision support, though explanations themselves are not the decisive driver for this improvement. The analysis reveals that the studies' risk of bias moderates the effect of explanations in AI, while the explanation type appears to play only a negligible role. Our findings contribute to the human computer interaction field by enhancing the understanding of human-XAI collaboration in DSS.