What Does Explainable AI Really Mean? A New Conceptualization of Perspectives
This work addresses the conceptual confusion in explainable AI for researchers across fields, but it is incremental as it builds on existing ideas.
The paper tackles the problem of defining explainable AI by characterizing three existing notions and proposing a fourth, based on a corpus analysis of research fields, to clarify perspectives without presenting specific numerical results.
We characterize three notions of explainable AI that cut across research fields: opaque systems that offer no insight into its algo- rithmic mechanisms; interpretable systems where users can mathemat- ically analyze its algorithmic mechanisms; and comprehensible systems that emit symbols enabling user-driven explanations of how a conclusion is reached. The paper is motivated by a corpus analysis of NIPS, ACL, COGSCI, and ICCV/ECCV paper titles showing differences in how work on explainable AI is positioned in various fields. We close by introducing a fourth notion: truly explainable systems, where automated reasoning is central to output crafted explanations without requiring human post processing as final step of the generative process.