Max-margin Deep Generative Models
This work addresses the limitation of DGMs in making accurate predictions, which is important for applications requiring both generative and discriminative capabilities, though it is incremental as it builds on existing DGM frameworks.
The paper tackles the problem of improving the discriminative ability of deep generative models (DGMs) while retaining their generative capability, resulting in max-margin deep generative models (mmDGMs) that significantly enhance prediction performance on MNIST and SVHN datasets and are competitive with state-of-the-art fully discriminative networks.
Deep generative models (DGMs) are effective on learning multilayered representations of complex data and performing inference of input data by exploring the generative ability. However, little work has been done on examining or empowering the discriminative ability of DGMs on making accurate predictions. This paper presents max-margin deep generative models (mmDGMs), which explore the strongly discriminative principle of max-margin learning to improve the discriminative power of DGMs, while retaining the generative capability. We develop an efficient doubly stochastic subgradient algorithm for the piecewise linear objective. Empirical results on MNIST and SVHN datasets demonstrate that (1) max-margin learning can significantly improve the prediction performance of DGMs and meanwhile retain the generative ability; and (2) mmDGMs are competitive to the state-of-the-art fully discriminative networks by employing deep convolutional neural networks (CNNs) as both recognition and generative models.