Generative Models For Deep Learning with Very Scarce Data
This addresses data scarcity in deep learning, which is a common bottleneck in domains with limited labeled data, but the approach is incremental as it applies existing generative methods to this problem.
The paper tackles the problem of deep learning failure under data scarcity by using Restricted Boltzmann Machines (RBM) and Variational Auto-encoders (VAE) as generative models to augment training sets, showing improved generalization over state-of-the-art techniques like semi-supervised learning with ladder networks and that RBM outperforms VAE in generating effective samples for classification.
The goal of this paper is to deal with a data scarcity scenario where deep learning techniques use to fail. We compare the use of two well established techniques, Restricted Boltzmann Machines and Variational Auto-encoders, as generative models in order to increase the training set in a classification framework. Essentially, we rely on Markov Chain Monte Carlo (MCMC) algorithms for generating new samples. We show that generalization can be improved comparing this methodology to other state-of-the-art techniques, e.g. semi-supervised learning with ladder networks. Furthermore, we show that RBM is better than VAE generating new samples for training a classifier with good generalization capabilities.