Extracting Rules from Neural Networks with Partial Interpretations
This work addresses interpretability in AI for researchers, but it is incremental as it applies existing methods to neural networks.
The paper tackles the problem of extracting Horn logic rules from neural networks using partial interpretations and Angluin's algorithm, and empirically evaluates this strategy.
We investigate the problem of extracting rules, expressed in Horn logic, from neural network models. Our work is based on the exact learning model, in which a learner interacts with a teacher (the neural network model) via queries in order to learn an abstract target concept, which in our case is a set of Horn rules. We consider partial interpretations to formulate the queries. These can be understood as a representation of the world where part of the knowledge regarding the truthiness of propositions is unknown. We employ Angluin s algorithm for learning Horn rules via queries and evaluate our strategy empirically.