NN2Rules: Extracting Rule List from Neural Networks
This addresses the need for interpretable AI models in domains where decision transparency is crucial, though it is incremental as it extends existing decompositional approaches to support more activation types.
The paper tackles the problem of making neural networks more interpretable by converting them into rule lists, and presents NN2Rules, an algorithm that achieves this with the same prediction accuracy as the original network.
We present an algorithm, NN2Rules, to convert a trained neural network into a rule list. Rule lists are more interpretable since they align better with the way humans make decisions. NN2Rules is a decompositional approach to rule extraction, i.e., it extracts a set of decision rules from the parameters of the trained neural network model. We show that the decision rules extracted have the same prediction as the neural network on any input presented to it, and hence the same accuracy. A key contribution of NN2Rules is that it allows hidden neuron behavior to be either soft-binary (eg. sigmoid activation) or rectified linear (ReLU) as opposed to existing decompositional approaches that were developed with the assumption of soft-binary activation.