Adaptive Scan Gibbs Sampler for Large Scale Inference Problems
This addresses computational bottlenecks in Bayesian inference for machine learning practitioners, though it appears incremental as an optimization of existing Gibbs sampling methods.
The authors tackled the problem of slow performance in large-scale online inference by developing an adaptive scan Gibbs sampler that optimizes update frequency through mini-batch size selection, demonstrating improvements over collapsed Gibbs samplers in Bayesian Lasso, DPMM, and LDA models.
For large scale on-line inference problems the update strategy is critical for performance. We derive an adaptive scan Gibbs sampler that optimizes the update frequency by selecting an optimum mini-batch size. We demonstrate performance of our adaptive batch-size Gibbs sampler by comparing it against the collapsed Gibbs sampler for Bayesian Lasso, Dirichlet Process Mixture Models (DPMM) and Latent Dirichlet Allocation (LDA) graphical models.