Unveiling the Decision-Making Process in Reinforcement Learning with Genetic Programming
This work addresses the need for interpretability in reinforcement learning for real-world applications, though it is incremental as it builds on existing methods with efficiency improvements.
The authors tackled the problem of incomprehensible predictions in reinforcement learning by proposing a genetic programming framework to generate interpretable explanations for trained agents' decisions, achieving comparable performance to the state of the art while requiring significantly less hardware resources and computation time.
Despite tremendous progress, machine learning and deep learning still suffer from incomprehensible predictions. Incomprehensibility, however, is not an option for the use of (deep) reinforcement learning in the real world, as unpredictable actions can seriously harm the involved individuals. In this work, we propose a genetic programming framework to generate explanations for the decision-making process of already trained agents by imitating them with programs. Programs are interpretable and can be executed to generate explanations of why the agent chooses a particular action. Furthermore, we conduct an ablation study that investigates how extending the domain-specific language by using library learning alters the performance of the method. We compare our results with the previous state of the art for this problem and show that we are comparable in performance but require much less hardware resources and computation time.