Causal Explanation for Reinforcement Learning: Quantifying State and Temporal Importance
This addresses the need for interpretable AI in RL, offering causal insights for users in fields like agriculture and robotics, though it appears incremental as it builds on existing explanation methods.
The paper tackled the problem of providing causal explanations for reinforcement learning policies by quantifying state and temporal importance, demonstrating advantages over associational methods in simulation studies across domains like crop irrigation and lunar lander.
Explainability plays an increasingly important role in machine learning. Furthermore, humans view the world through a causal lens and thus prefer causal explanations over associational ones. Therefore, in this paper, we develop a causal explanation mechanism that quantifies the causal importance of states on actions and such importance over time. We also demonstrate the advantages of our mechanism over state-of-the-art associational methods in terms of RL policy explanation through a series of simulation studies, including crop irrigation, Blackjack, collision avoidance, and lunar lander.