A Framework of Explanation Generation toward Reliable Autonomous Robots
This addresses the need for reliable autonomous robots by improving human trust through explainable AI, though it is incremental as it builds on existing MDP and cognitive science principles.
The paper tackles the problem of increasing user trust in autonomous robots by proposing a framework for generating explanations of state transitions in Markov decision processes, with experiments showing that the method produces minimal, understandable explanations and receives high evaluations for action reasoning.
To realize autonomous collaborative robots, it is important to increase the trust that users have in them. Toward this goal, this paper proposes an algorithm which endows an autonomous agent with the ability to explain the transition from the current state to the target state in a Markov decision process (MDP). According to cognitive science, to generate an explanation that is acceptable to humans, it is important to present the minimum information necessary to sufficiently understand an event. To meet this requirement, this study proposes a framework for identifying important elements in the decision-making process using a prediction model for the world and generating explanations based on these elements. To verify the ability of the proposed method to generate explanations, we conducted an experiment using a grid environment. It was inferred from the result of a simulation experiment that the explanation generated using the proposed method was composed of the minimum elements important for understanding the transition from the current state to the target state. Furthermore, subject experiments showed that the generated explanation was a good summary of the process of state transition, and that a high evaluation was obtained for the explanation of the reason for an action.