Stakeholders in Explainable AI
This work clarifies foundational concepts in explainable AI, which is crucial for researchers and practitioners to align on goals and methods, though it is incremental in nature.
The paper addresses the lack of consensus on definitions of 'explainable' and 'interpretable' in AI by identifying distinct stakeholder communities with different intents and requirements, using software engineering and epistemological distinctions to analyze their concerns.
There is general consensus that it is important for artificial intelligence (AI) and machine learning systems to be explainable and/or interpretable. However, there is no general consensus over what is meant by 'explainable' and 'interpretable'. In this paper, we argue that this lack of consensus is due to there being several distinct stakeholder communities. We note that, while the concerns of the individual communities are broadly compatible, they are not identical, which gives rise to different intents and requirements for explainability/interpretability. We use the software engineering distinction between validation and verification, and the epistemological distinctions between knowns/unknowns, to tease apart the concerns of the stakeholder communities and highlight the areas where their foci overlap or diverge. It is not the purpose of the authors of this paper to 'take sides' - we count ourselves as members, to varying degrees, of multiple communities - but rather to help disambiguate what stakeholders mean when they ask 'Why?' of an AI.