Explainable and Human-Grounded AI for Decision Support Systems: The Theory of Epistemic Quasi-Partnerships
This work addresses the need for explainable AI in decision support systems to enhance trust and accuracy for human users, presenting a theoretical framework rather than incremental improvements.
The paper tackles the problem of making AI decision support systems (AI-DSS) more ethical and explainable by proposing the RCC approach, which provides human-grounded explanations including reasons, counterfactuals, and confidence, and introduces the theory of epistemic quasi-partnerships (EQP) to explain empirical evidence and offer ethical guidance.
In the context of AI decision support systems (AI-DSS), we argue that meeting the demands of ethical and explainable AI (XAI) is about developing AI-DSS to provide human decision-makers with three types of human-grounded explanations: reasons, counterfactuals, and confidence, an approach we refer to as the RCC approach. We begin by reviewing current empirical XAI literature that investigates the relationship between various methods for generating model explanations (e.g., LIME, SHAP, Anchors), the perceived trustworthiness of the model, and end-user accuracy. We demonstrate how current theories about what constitutes good human-grounded reasons either do not adequately explain this evidence or do not offer sound ethical advice for development. Thus, we offer a novel theory of human-machine interaction: the theory of epistemic quasi-partnerships (EQP). Finally, we motivate adopting EQP and demonstrate how it explains the empirical evidence, offers sound ethical advice, and entails adopting the RCC approach.