Accelerated Parallel Non-conjugate Sampling for Bayesian Non-parametric Models
This work addresses computational bottlenecks in Bayesian nonparametric inference for researchers and practitioners, though it is incremental as it builds on existing MCMC methods.
The paper tackles the slow mixing problem in Bayesian nonparametric latent feature models by introducing a data-driven feature proposal mechanism and a parallel approximate inference strategy, achieving faster convergence to the posterior distribution.
Inference of latent feature models in the Bayesian nonparametric setting is generally difficult, especially in high dimensional settings, because it usually requires proposing features from some prior distribution. In special cases, where the integration is tractable, we can sample new feature assignments according to a predictive likelihood. We present a novel method to accelerate the mixing of latent variable model inference by proposing feature locations based on the data, as opposed to the prior. First, we introduce an accelerated feature proposal mechanism that we show is a valid MCMC algorithm for posterior inference. Next, we propose an approximate inference strategy to perform accelerated inference in parallel. A two-stage algorithm that combines the two approaches provides a computationally attractive method that can quickly reach local convergence to the posterior distribution of our model, while allowing us to exploit parallelization.