Backpropagation Neural Tree
This work addresses the need for more efficient and biologically inspired neural models in machine learning, though it appears incremental as it builds on existing gradient descent optimizers and neural network concepts.
The authors tackled the problem of creating a biologically plausible, computationally efficient neural model by proposing the Backpropagation Neural Tree (BNeuralT), which is a single neuron neural tree with dendritic nonlinearities, and the result is a high-performing and parsimonious model that balances complexity with descriptive ability across classification, regression, and pattern recognition tasks.
We propose a novel algorithm called Backpropagation Neural Tree (BNeuralT), which is a stochastic computational dendritic tree. BNeuralT takes random repeated inputs through its leaves and imposes dendritic nonlinearities through its internal connections like a biological dendritic tree would do. Considering the dendritic-tree like plausible biological properties, BNeuralT is a single neuron neural tree model with its internal sub-trees resembling dendritic nonlinearities. BNeuralT algorithm produces an ad hoc neural tree which is trained using a stochastic gradient descent optimizer like gradient descent (GD), momentum GD, Nesterov accelerated GD, Adagrad, RMSprop, or Adam. BNeuralT training has two phases, each computed in a depth-first search manner: the forward pass computes neural tree's output in a post-order traversal, while the error backpropagation during the backward pass is performed recursively in a pre-order traversal. A BNeuralT model can be considered a minimal subset of a neural network (NN), meaning it is a "thinned" NN whose complexity is lower than an ordinary NN. Our algorithm produces high-performing and parsimonious models balancing the complexity with descriptive ability on a wide variety of machine learning problems: classification, regression, and pattern recognition.