Deep Learning of Part-based Representation of Data Using Sparse Autoencoders with Nonnegativity Constraints
This incremental method enhances feature learning for part-based representation in deep learning, benefiting researchers in computer vision and natural language processing.
The authors introduced a nonnegativity constraint algorithm (NCAE) for deep autoencoders to learn part-based data representations, showing improved sparsity and reconstruction quality over traditional sparse autoencoders and Nonnegative Matrix Factorization on three image and one text dataset.
We demonstrate a new deep learning autoencoder network, trained by a nonnegativity constraint algorithm (NCAE), that learns features which show part-based representation of data. The learning algorithm is based on constraining negative weights. The performance of the algorithm is assessed based on decomposing data into parts and its prediction performance is tested on three standard image data sets and one text dataset. The results indicate that the nonnegativity constraint forces the autoencoder to learn features that amount to a part-based representation of data, while improving sparsity and reconstruction quality in comparison with the traditional sparse autoencoder and Nonnegative Matrix Factorization. It is also shown that this newly acquired representation improves the prediction performance of a deep neural network.