CODEX: A Cluster-Based Method for Explainable Reinforcement Learning
This addresses the need for explainable AI in high-risk domains like reinforcement learning, though it appears incremental by extending NLP techniques to RL.
The paper tackles the problem of explaining reinforcement learning agent actions to enable adoption in high-risk applications, by introducing CODEX, a method that uses semantic clustering to summarize agent behavior, with experiments on MiniGrid and StarCraft II showing retention of temporal and entity information in clusters.
Despite the impressive feats demonstrated by Reinforcement Learning (RL), these algorithms have seen little adoption in high-risk, real-world applications due to current difficulties in explaining RL agent actions and building user trust. We present Counterfactual Demonstrations for Explanation (CODEX), a method that incorporates semantic clustering, which can effectively summarize RL agent behavior in the state-action space. Experimentation on the MiniGrid and StarCraft II gaming environments reveals the semantic clusters retain temporal as well as entity information, which is reflected in the constructed summary of agent behavior. Furthermore, clustering the discrete+continuous game-state latent representations identifies the most crucial episodic events, demonstrating a relationship between the latent and semantic spaces. This work contributes to the growing body of work that strives to unlock the power of RL for widespread use by leveraging and extending techniques from Natural Language Processing.