Inherently Explainable Reinforcement Learning in Natural Language
This addresses the need for explainable AI in complex, long-term decision-making tasks like text-based games, offering both local and post-hoc explanations, though it is incremental in applying existing methods to enhance explainability.
The paper tackled the problem of creating a reinforcement learning agent that is inherently explainable in natural language, using a Hierarchically Explainable Reinforcement Learning agent (HEX-RL) in text-based games. The result showed that this agent provided significantly improved explanations over strong baselines, as rated by human participants, while matching state-of-the-art task performance.
We focus on the task of creating a reinforcement learning agent that is inherently explainable -- with the ability to produce immediate local explanations by thinking out loud while performing a task and analyzing entire trajectories post-hoc to produce causal explanations. This Hierarchically Explainable Reinforcement Learning agent (HEX-RL), operates in Interactive Fictions, text-based game environments in which an agent perceives and acts upon the world using textual natural language. These games are usually structured as puzzles or quests with long-term dependencies in which an agent must complete a sequence of actions to succeed -- providing ideal environments in which to test an agent's ability to explain its actions. Our agent is designed to treat explainability as a first-class citizen, using an extracted symbolic knowledge graph-based state representation coupled with a Hierarchical Graph Attention mechanism that points to the facts in the internal graph representation that most influenced the choice of actions. Experiments show that this agent provides significantly improved explanations over strong baselines, as rated by human participants generally unfamiliar with the environment, while also matching state-of-the-art task performance.