Semi-Supervised Learning with the Deep Rendering Mixture Model
This provides a unified framework for supervised, unsupervised, and semi-supervised learning, addressing the high cost of labeled data acquisition for machine learning practitioners.
The paper tackled semi-supervised learning by developing an EM algorithm for the Deep Rendering Mixture Model (DRMM), incorporating a non-negativity constraint and variational inference term, achieving state-of-the-art performance on MNIST and SVHN and competitive results on CIFAR10.
Semi-supervised learning algorithms reduce the high cost of acquiring labeled training data by using both labeled and unlabeled data during learning. Deep Convolutional Networks (DCNs) have achieved great success in supervised tasks and as such have been widely employed in the semi-supervised learning. In this paper we leverage the recently developed Deep Rendering Mixture Model (DRMM), a probabilistic generative model that models latent nuisance variation, and whose inference algorithm yields DCNs. We develop an EM algorithm for the DRMM to learn from both labeled and unlabeled data. Guided by the theory of the DRMM, we introduce a novel non-negativity constraint and a variational inference term. We report state-of-the-art performance on MNIST and SVHN and competitive results on CIFAR10. We also probe deeper into how a DRMM trained in a semi-supervised setting represents latent nuisance variation using synthetically rendered images. Taken together, our work provides a unified framework for supervised, unsupervised, and semi-supervised learning.