Minimalistic Explanations: Capturing the Essence of Decisions
This work addresses the need for interpretable AI for users, but it is incremental as it builds on existing explanation methods like LIME and is limited to pilot-scale experiments.
The study tackled the problem of opaque machine learning models by testing minimalistic post-hoc explanations for image classifications, finding that participants identified explained objects with significantly higher accuracy (75.64% vs. 18.52%) for correct decisions and assigned 79% higher trust ratings to human-generated explanations.
The use of complex machine learning models can make systems opaque to users. Machine learning research proposes the use of post-hoc explanations. However, it is unclear if they give users insights into otherwise uninterpretable models. One minimalistic way of explaining image classifications by a deep neural network is to show only the areas that were decisive for the assignment of a label. In a pilot study, 20 participants looked at 14 of such explanations generated either by a human or the LIME algorithm. For explanations of correct decisions, they identified the explained object with significantly higher accuracy (75.64% vs. 18.52%). We argue that this shows that explanations can be very minimalistic while retaining the essence of a decision, but the decision-making contexts that can be conveyed in this manner is limited. Finally, we found that explanations are unique to the explainer and human-generated explanations were assigned 79% higher trust ratings. As a starting point for further studies, this work shares our first insights into quality criteria of post-hoc explanations.