Nonnegative autoencoder with simplified random neural network
This work addresses the need for efficient nonnegative autoencoders in image and real-world data processing, though it appears incremental as it combines existing methods.
The paper tackled the problem of designing nonnegative autoencoders by integrating spiking Random Neural Networks with deep learning architectures and NMF-inspired training, achieving high learning and recognition accuracy on datasets like MNIST, Yale face, and CIFAR-10.
This paper proposes new nonnegative (shallow and multi-layer) autoencoders by combining the spiking Random Neural Network (RNN) model, the network architecture typical used in deep-learning area and the training technique inspired from nonnegative matrix factorization (NMF). The shallow autoencoder is a simplified RNN model, which is then stacked into a multi-layer architecture. The learning algorithm is based on the weight update rules in NMF, subject to the nonnegative probability constraints of the RNN. The autoencoders equipped with this learning algorithm are tested on typical image datasets including the MNIST, Yale face and CIFAR-10 datasets, and also using 16 real-world datasets from different areas. The results obtained through these tests yield the desired high learning and recognition accuracy. Also, numerical simulations of the stochastic spiking behavior of this RNN auto encoder, show that it can be implemented in a highly-distributed manner.