Automatic Induction of Neural Network Decision Tree Algorithms
This work addresses the challenge of optimizing decision tree architectures for machine learning practitioners, though it appears incremental as it builds on existing ensemble and Bayesian methods.
The paper tackles the problem of automatically inducing non-greedy decision tree architectures using neural networks, with results showing improved performance over fixed hyperparameter models for decision trees and forests.
This work presents an approach to automatically induction for non-greedy decision trees constructed from neural network architecture. This construction can be used to transfer weights when growing or pruning a decision tree, allowing non-greedy decision tree algorithms to automatically learn and adapt to the ideal architecture. In this work, we examine the underpinning ideas within ensemble modelling and Bayesian model averaging which allow our neural network to asymptotically approach the ideal architecture through weights transfer. Experimental results demonstrate that this approach improves models over fixed set of hyperparameters for decision tree models and decision forest models.