Backward-Forward Algorithm: An Improvement towards Extreme Learning Machine
This work addresses a specific bottleneck in neural network training for researchers and practitioners, offering an incremental improvement over existing methods.
The authors tackled the problem of extreme learning machines requiring many hidden nodes, which can lead to memorization rather than generalization, by proposing a supervised method that uses Moore-Penrose approximation to determine input and output weights in two epochs. The result is a technique that reduces iterations compared to back-propagation and requires fewer hidden units than extreme learning machines for generalization.
The extreme learning machine needs a large number of hidden nodes to generalize a single hidden layer neural network for a given training data-set. The need for more number of hidden nodes suggests that the neural-network is memorizing rather than generalizing the model. Hence, a supervised learning method is described here that uses Moore-Penrose approximation to determine both input-weight and output-weight in two epochs, namely, backward-pass and forward-pass. The proposed technique has an advantage over the back-propagation method in terms of iterations required and is superior to the extreme learning machine in terms of the number of hidden units necessary for generalization.