Explainable Artificial Intelligence (XAI) for Increasing User Trust in Deep Reinforcement Learning Driven Autonomous Systems
This addresses the issue of trust for users of autonomous systems, but it is incremental as it applies existing XAI concepts to deep RL.
The paper tackled the problem of user trust in deep reinforcement learning systems by developing an explainable AI framework that provides graphical and narrative explanations. The results showed a statistically significant increase in user trust and acceptance when explanations were provided compared to when they were not.
We consider the problem of providing users of deep Reinforcement Learning (RL) based systems with a better understanding of when their output can be trusted. We offer an explainable artificial intelligence (XAI) framework that provides a three-fold explanation: a graphical depiction of the systems generalization and performance in the current game state, how well the agent would play in semantically similar environments, and a narrative explanation of what the graphical information implies. We created a user-interface for our XAI framework and evaluated its efficacy via a human-user experiment. The results demonstrate a statistically significant increase in user trust and acceptance of the AI system with explanation, versus the AI system without explanation.