Autoencoder based Randomized Learning of Feedforward Neural Networks for Regression
This work addresses the inefficiency of gradient-based training for feedforward neural networks in regression, but it is incremental as it adapts an existing autoencoder method from classification to regression with limited success.
The authors tackled the problem of applying autoencoder-based randomized learning to regression tasks, finding that despite improvements to control random weights and determine biases, the method did not outperform other recent randomized learning methods in fitting accuracy and was more complex.
Feedforward neural networks are widely used as universal predictive models to fit data distribution. Common gradient-based learning, however, suffers from many drawbacks making the training process ineffective and time-consuming. Alternative randomized learning does not use gradients but selects hidden node parameters randomly. This makes the training process extremely fast. However, the problem in randomized learning is how to determine the random parameters. A recently proposed method uses autoencoders for unsupervised parameter learning. This method showed superior performance on classification tasks. In this work, we apply this method to regression problems, and, finding that it has some drawbacks, we show how to improve it. We propose a learning method of autoencoders that controls the produced random weights. We also propose how to determine the biases of hidden nodes. We empirically compare autoencoder based learning with other randomized learning methods proposed recently for regression and find that despite the proposed improvement of the autoencoder based learning, it does not outperform its competitors in fitting accuracy. Moreover, the method is much more complex than its competitors.