Using Program Induction to Interpret Transition System Dynamics
This addresses the need for interpretable models in AI, particularly for understanding complex systems, but is incremental as it builds on existing program induction techniques.
The authors tackled the problem of explaining black-box phenomena by learning interpretable LISP-like programs from data traces, using a method called the π-machine, and showed it can efficiently induce programs for system identification and DQN agent behavior.
Explaining and reasoning about processes which underlie observed black-box phenomena enables the discovery of causal mechanisms, derivation of suitable abstract representations and the formulation of more robust predictions. We propose to learn high level functional programs in order to represent abstract models which capture the invariant structure in the observed data. We introduce the $π$-machine (program-induction machine) -- an architecture able to induce interpretable LISP-like programs from observed data traces. We propose an optimisation procedure for program learning based on backpropagation, gradient descent and A* search. We apply the proposed method to two problems: system identification of dynamical systems and explaining the behaviour of a DQN agent. Our results show that the $π$-machine can efficiently induce interpretable programs from individual data traces.