Designing Interpretable Approximations to Deep Reinforcement Learning
This work addresses the need for verifiable and deployable AI in resource-constrained or safety-critical domains, though it is incremental as it builds on existing model compression and distillation techniques.
The paper tackles the problem of making deep reinforcement learning models more interpretable and efficient by approximating them with simpler models like decision trees and kernel machines, while maintaining performance on benchmark tasks.
In an ever expanding set of research and application areas, deep neural networks (DNNs) set the bar for algorithm performance. However, depending upon additional constraints such as processing power and execution time limits, or requirements such as verifiable safety guarantees, it may not be feasible to actually use such high-performing DNNs in practice. Many techniques have been developed in recent years to compress or distill complex DNNs into smaller, faster or more understandable models and controllers. This work seeks to identify reduced models that not only preserve a desired performance level, but also, for example, succinctly explain the latent knowledge represented by a DNN. We illustrate the effectiveness of the proposed approach on the evaluation of decision tree variants and kernel machines in the context of benchmark reinforcement learning tasks.