Growing axons: greedy learning of neural networks with application to function approximation
This work addresses function approximation in machine learning, but appears incremental as it builds on existing greedy learning approaches.
The authors tackled the problem of learning deep neural networks by introducing a greedy method that adds one neuron at a time, resulting in more accurate function approximants for several model problems.
We propose a new method for learning deep neural network models that is based on a greedy learning approach: we add one basis function at a time, and a new basis function is generated as a non-linear activation function applied to a linear combination of the previous basis functions. Such a method (growing deep neural network by one neuron at a time) allows us to compute much more accurate approximants for several model problems in function approximation.