Improving understanding and trust in AI: How users benefit from interval-based counterfactual explanations
This addresses the problem of improving user trust and understanding in AI systems, though it is incremental as it builds on existing explanation methods.
The study tackled the lack of experimental user studies on post-hoc explanations for black-box models by evaluating how single point and interval-based counterfactual explanations affect model understanding and trust, finding that interval explanations were superior in increasing both compared to other types.
Experimental user studies evaluating the effectiveness of different subtypes of post-hoc explanations for black-box models are largely nonexistent. Therefore, the aim of this study was to investigate and evaluate how different types of counterfactual explanations, namely single point explanations and interval-based explanations, affect both model understanding and (demonstrated) trust. We conducted an online user study using a within-subjects experimental design, where the experimental arms were (i) no explanation (control), (ii) feature importance scores, (iii) point counterfactual explanations, and (iv) interval counterfactual explanations. Our results clearly show the superiority of interval explanations over other tested explanation types in increasing both model understanding and demonstrated trust in the AI. We could not support findings of some previous studies showing an effect of point counterfactual explanations compared to the control group. Our results further highlight the role individual differences in, for example, cognitive style or personality, in explanation effectiveness.