Should We Trust (X)AI? Design Dimensions for Structured Experimental Evaluations
This work addresses the challenge of inconsistent and fragmented evaluation practices in XAI research, which is crucial for researchers and practitioners aiming to develop trustworthy AI systems, though it is incremental in building a framework rather than introducing a new method.
The paper tackles the problem of evaluating explainable AI (XAI) methods by systematically deriving design dimensions for structured experimental evaluations, resulting in a conceptual model that characterizes bias sources and trust-building to enable holistic assessments in real-world scenarios.
This paper systematically derives design dimensions for the structured evaluation of explainable artificial intelligence (XAI) approaches. These dimensions enable a descriptive characterization, facilitating comparisons between different study designs. They further structure the design space of XAI, converging towards a precise terminology required for a rigorous study of XAI. Our literature review differentiates between comparative studies and application papers, revealing methodological differences between the fields of machine learning, human-computer interaction, and visual analytics. Generally, each of these disciplines targets specific parts of the XAI process. Bridging the resulting gaps enables a holistic evaluation of XAI in real-world scenarios, as proposed by our conceptual model characterizing bias sources and trust-building. Furthermore, we identify and discuss the potential for future work based on observed research gaps that should lead to better coverage of the proposed model.