Defining Explainable AI for Requirements Analysis
This work addresses the need for tailored explainability in AI applications, but it is incremental as it builds on existing literature by focusing on categorization without introducing new methods.
The paper tackles the problem of defining explanatory requirements for AI systems to ensure trust, by presenting three dimensions (Source, Depth, and Scope) to categorize these requirements and match them with ML techniques' capabilities.
Explainable Artificial Intelligence (XAI) has become popular in the last few years. The Artificial Intelligence (AI) community in general, and the Machine Learning (ML) community in particular, is coming to the realisation that in many applications, for AI to be trusted, it must not only demonstrate good performance in its decisionmaking, but it also must explain these decisions and convince us that it is making the decisions for the right reasons. However, different applications have different requirements on the information required of the underlying AI system in order to convince us that it is worthy of our trust. How do we define these requirements? In this paper, we present three dimensions for categorising the explanatory requirements of different applications. These are Source, Depth and Scope. We focus on the problem of matching up the explanatory requirements of different applications with the capabilities of underlying ML techniques to provide them. We deliberately avoid including aspects of explanation that are already well-covered by the existing literature and we focus our discussion on ML although the principles apply to AI more broadly.