On the Variational Posterior of Dirichlet Process Deep Latent Gaussian Mixture Models
This work addresses a bottleneck in nonparametric Bayesian deep learning for researchers, though it is incremental as it builds on existing methods to enable DP coupling.
The paper tackled the challenge of integrating Dirichlet Process priors with deep latent Gaussian models by developing a variational posterior that allows closed-form updates for cluster parameters and beta distributions, enabling the use of the reparameterization trick. It demonstrated competitive performance in semi-supervised learning and generated realistic samples on benchmark datasets.
Thanks to the reparameterization trick, deep latent Gaussian models have shown tremendous success recently in learning latent representations. The ability to couple them however with nonparamet-ric priors such as the Dirichlet Process (DP) hasn't seen similar success due to its non parameteriz-able nature. In this paper, we present an alternative treatment of the variational posterior of the Dirichlet Process Deep Latent Gaussian Mixture Model (DP-DLGMM), where we show that the prior cluster parameters and the variational posteriors of the beta distributions and cluster hidden variables can be updated in closed-form. This leads to a standard reparameterization trick on the Gaussian latent variables knowing the cluster assignments. We demonstrate our approach on standard benchmark datasets, we show that our model is capable of generating realistic samples for each cluster obtained, and manifests competitive performance in a semi-supervised setting.