Deep Neural Decision Trees
This work addresses the challenge of making neural networks more interpretable and effective for tabular data, which is incremental as it builds on existing tree and neural network methods.
The authors tackled the problem of applying neural networks to tabular data by proposing Deep Neural Decision Trees (DNDT), which combine the interpretability of trees with the training efficiency of neural networks, achieving competitive performance on several tabular datasets.
Deep neural networks have been proven powerful at processing perceptual data, such as images and audio. However for tabular data, tree-based models are more popular. A nice property of tree-based models is their natural interpretability. In this work, we present Deep Neural Decision Trees (DNDT) -- tree models realised by neural networks. A DNDT is intrinsically interpretable, as it is a tree. Yet as it is also a neural network (NN), it can be easily implemented in NN toolkits, and trained with gradient descent rather than greedy splitting. We evaluate DNDT on several tabular datasets, verify its efficacy, and investigate similarities and differences between DNDT and vanilla decision trees. Interestingly, DNDT self-prunes at both split and feature-level.