Explaining Transition Systems through Program Induction
This enables interpretable modeling of complex systems for researchers and practitioners, though it is incremental as it builds on existing program induction methods.
The authors tackled the problem of explaining black-box processes by learning interpretable LISP-like programs from observed data traces, achieving efficient program induction across tasks like system identification and agent behavior explanation.
Explaining and reasoning about processes which underlie observed black-box phenomena enables the discovery of causal mechanisms, derivation of suitable abstract representations and the formulation of more robust predictions. We propose to learn high level functional programs in order to represent abstract models which capture the invariant structure in the observed data. We introduce the $π$-machine (program-induction machine) -- an architecture able to induce interpretable LISP-like programs from observed data traces. We propose an optimisation procedure for program learning based on backpropagation, gradient descent and A* search. We apply the proposed method to three problems: system identification of dynamical systems, explaining the behaviour of a DQN agent and learning by demonstration in a human-robot interaction scenario. Our experimental results show that the $π$-machine can efficiently induce interpretable programs from individual data traces.