Bottleneck Conditional Density Estimation
This addresses the problem of training deep generative models for high-dimensional conditional density estimation, which is incremental with a novel hybrid training approach.
The paper tackles high-dimensional conditional density estimation by introducing the Bottleneck Conditional Density Estimator (BCDE), a variant of CVAE with stochastic bottleneck layers, and a hybrid training method blending conditional and joint generative models. The method achieves competitive results on MNIST quadrant prediction in supervised settings and sets new benchmarks for semi-supervised learning on MNIST, SVHN, and CelebA.
We introduce a new framework for training deep generative models for high-dimensional conditional density estimation. The Bottleneck Conditional Density Estimator (BCDE) is a variant of the conditional variational autoencoder (CVAE) that employs layer(s) of stochastic variables as the bottleneck between the input $x$ and target $y$, where both are high-dimensional. Crucially, we propose a new hybrid training method that blends the conditional generative model with a joint generative model. Hybrid blending is the key to effective training of the BCDE, which avoids overfitting and provides a novel mechanism for leveraging unlabeled data. We show that our hybrid training procedure enables models to achieve competitive results in the MNIST quadrant prediction task in the fully-supervised setting, and sets new benchmarks in the semi-supervised regime for MNIST, SVHN, and CelebA.