XDQN: Inherently Interpretable DQN through Mimicking
This addresses the problem of limited interpretability in DRL for real-world operational settings like air traffic management, though it is incremental as it builds on existing interpretable box design paradigms.
The paper tackles the lack of explainability in deep reinforcement learning (DRL) by proposing XDQN, an inherently interpretable variation of DQN that uses an interpretable policy model trained through mimicking, achieving performance similar to DQN in multi-agent air traffic management scenarios.
Although deep reinforcement learning (DRL) methods have been successfully applied in challenging tasks, their application in real-world operational settings is challenged by methods' limited ability to provide explanations. Among the paradigms for explainability in DRL is the interpretable box design paradigm, where interpretable models substitute inner constituent models of the DRL method, thus making the DRL method "inherently" interpretable. In this paper we explore this paradigm and we propose XDQN, an explainable variation of DQN, which uses an interpretable policy model trained through mimicking. XDQN is challenged in a complex, real-world operational multi-agent problem, where agents are independent learners solving congestion problems. Specifically, XDQN is evaluated in three MARL scenarios, pertaining to the demand-capacity balancing problem of air traffic management. XDQN achieves performance similar to that of DQN, while its abilities to provide global models' interpretations and interpretations of local decisions are demonstrated.