Deep Learning of Nonnegativity-Constrained Autoencoders for Enhanced Understanding of Data
This work addresses the interpretability bottleneck in deep learning for researchers and practitioners, though it is incremental as it builds on existing autoencoder methods.
The paper tackled the problem of limited interpretability in deep unsupervised feature extractors by introducing a Nonnegativity Constrained Autoencoder (NCSAE) with L1 and L2 regularization, resulting in more understandable network structures with minimal accuracy loss on datasets like MNIST, NORB, and Reuters.
Unsupervised feature extractors are known to perform an efficient and discriminative representation of data. Insight into the mappings they perform and human ability to understand them, however, remain very limited. This is especially prominent when multilayer deep learning architectures are used. This paper demonstrates how to remove these bottlenecks within the architecture of Nonnegativity Constrained Autoencoder (NCSAE). It is shown that by using both L1 and L2 regularization that induce nonnegativity of weights, most of the weights in the network become constrained to be nonnegative thereby resulting into a more understandable structure with minute deterioration in classification accuracy. Also, this proposed approach extracts features that are more sparse and produces additional output layer sparsification. The method is analyzed for accuracy and feature interpretation on the MNIST data, the NORB normalized uniform object data, and the Reuters text categorization dataset.