Abstracted Trajectory Visualization for Explainability in Reinforcement Learning
This addresses the challenge of enabling non-RL experts to participate in RL design discussions for human-AI coexistence, though it appears incremental as it builds on existing XAI concepts.
The paper tackles the problem of making reinforcement learning (RL) models explainable to non-RL experts by proposing abstracted trajectory visualization, with early results suggesting it helps users infer RL behavior patterns.
Explainable AI (XAI) has demonstrated the potential to help reinforcement learning (RL) practitioners to understand how RL models work. However, XAI for users who do not have RL expertise (non-RL experts), has not been studied sufficiently. This results in a difficulty for the non-RL experts to participate in the fundamental discussion of how RL models should be designed for an incoming society where humans and AI coexist. Solving such a problem would enable RL experts to communicate with the non-RL experts in producing machine learning solutions that better fit our society. We argue that abstracted trajectories, that depicts transitions between the major states of the RL model, will be useful for non-RL experts to build a mental model of the agents. Our early results suggest that by leveraging a visualization of the abstracted trajectories, users without RL expertise are able to infer the behavior patterns of RL.