Hybrid VAE: Improving Deep Generative Models using Partial Observations
This addresses the challenge of expensive labeled data in computer vision applications by enabling more efficient training with partial observations, though it is incremental as it builds on existing generative models.
The paper tackled the problem of limited labeled data in deep generative models by proposing a framework that leverages both labeled and unlabeled data, showing that this combination allows models trained on few labeled samples to match the performance of fully-supervised models on larger datasets, with validation on three large-scale datasets including MultiPIE, CelebA, and NYU Hand Pose.
Deep neural network models trained on large labeled datasets are the state-of-the-art in a large variety of computer vision tasks. In many applications, however, labeled data is expensive to obtain or requires a time consuming manual annotation process. In contrast, unlabeled data is often abundant and available in large quantities. We present a principled framework to capitalize on unlabeled data by training deep generative models on both labeled and unlabeled data. We show that such a combination is beneficial because the unlabeled data acts as a data-driven form of regularization, allowing generative models trained on few labeled samples to reach the performance of fully-supervised generative models trained on much larger datasets. We call our method Hybrid VAE (H-VAE) as it contains both the generative and the discriminative parts. We validate H-VAE on three large-scale datasets of different modalities: two face datasets: (MultiPIE, CelebA) and a hand pose dataset (NYU Hand Pose). Our qualitative visualizations further support improvements achieved by using partial observations.