Accounts, Accountability and Agency for Safe and Ethical AI
This addresses the problem of making AI interpretable and accountable in high-stakes domains like healthcare, but it is incremental as it builds on existing xAI concepts without introducing new technical solutions.
The paper examines explainable AI (xAI) in collaborative decision-making contexts, using a case study of mammography screening to illustrate challenges in interpreting AI behavior, and proposes a future research program to advance xAI requirements for safe and ethical AI.
We examine the problem of explainable AI (xAI) and explore what delivering xAI means in practice, particularly in contexts that involve formal or informal and ad-hoc collaboration where agency and accountability in decision-making are achieved and sustained interactionally. We use an example from an earlier study of collaborative decision-making in screening mammography and the difficulties users faced when trying to interpret the behavior of an AI tool to illustrate the challenges of delivering usable and effective xAI. We conclude by setting out a study programme for future research to help advance our understanding of xAI requirements for safe and ethical AI.