Distilling a Neural Network Into a Soft Decision Tree
This addresses the interpretability issue in neural networks for classification tasks, making decisions easier to explain, though it is incremental as it builds on existing distillation and tree methods.
The paper tackles the problem of explaining neural network decisions by distilling a trained neural network into a soft decision tree, which generalizes better than trees learned directly from training data.
Deep neural networks have proved to be a very effective way to perform classification tasks. They excel when the input data is high dimensional, the relationship between the input and the output is complicated, and the number of labeled training examples is large. But it is hard to explain why a learned network makes a particular classification decision on a particular test case. This is due to their reliance on distributed hierarchical representations. If we could take the knowledge acquired by the neural net and express the same knowledge in a model that relies on hierarchical decisions instead, explaining a particular decision would be much easier. We describe a way of using a trained neural net to create a type of soft decision tree that generalizes better than one learned directly from the training data.