Sampling for Bayesian Mixture Models: MCMC with Polynomial-Time Mixing
This addresses the challenge of slow mixing in MCMC for Bayesian mixture models, which is incremental as it focuses on a specific algorithm and theoretical analysis for a particular model class.
The paper tackles the problem of sampling from the power posterior in Bayesian Gaussian mixture models, which is non-log-concave and multi-modal, by introducing the Reflected Metropolis-Hastings Random Walk algorithm and proving its mixing time is bounded as $d^{1.5}(d + \Vert θ_{0} \Vert^2)^{4.5}$ for symmetric two-component mixtures with sample size $n$ of order $d (d + \Vert θ_{0} \Vert^2)$ without separation conditions.
We study the problem of sampling from the power posterior distribution in Bayesian Gaussian mixture models, a robust version of the classical posterior. This power posterior is known to be non-log-concave and multi-modal, which leads to exponential mixing times for some standard MCMC algorithms. We introduce and study the Reflected Metropolis-Hastings Random Walk (RMRW) algorithm for sampling. For symmetric two-component Gaussian mixtures, we prove that its mixing time is bounded as $d^{1.5}(d + \Vert θ_{0} \Vert^2)^{4.5}$ as long as the sample size $n$ is of the order $d (d + \Vert θ_{0} \Vert^2)$. Notably, this result requires no conditions on the separation of the two means. En route to proving this bound, we establish some new results of possible independent interest that allow for combining Poincaré inequalities for conditional and marginal densities.