Backprojection for Training Feedforward Neural Networks in the Input and Feature Spaces
This work addresses the need for alternative training algorithms in neural networks to gain insights, though it appears incremental as it builds on existing projection methods without broad application.
The paper tackles the problem of training feedforward neural networks by proposing a new algorithm based on projection and reconstruction, which is reported to be faster than backpropagation, with experiments on synthetic datasets demonstrating its effectiveness.
After the tremendous development of neural networks trained by backpropagation, it is a good time to develop other algorithms for training neural networks to gain more insights into networks. In this paper, we propose a new algorithm for training feedforward neural networks which is fairly faster than backpropagation. This method is based on projection and reconstruction where, at every layer, the projected data and reconstructed labels are forced to be similar and the weights are tuned accordingly layer by layer. The proposed algorithm can be used for both input and feature spaces, named as backprojection and kernel backprojection, respectively. This algorithm gives an insight to networks with a projection-based perspective. The experiments on synthetic datasets show the effectiveness of the proposed method.