Measuring the non-asymptotic convergence of sequential Monte Carlo samplers using probabilistic programming
This addresses the challenge of error quantification in approximate inference for researchers and practitioners using SMC samplers, though it is incremental as it builds on existing assumptions about gold-standard samplers.
The paper tackles the problem of quantifying approximation error in sequential Monte Carlo samplers by developing a method to upper-bound the symmetric KL divergence between the sampler's output and the target posterior distribution, with experiments showing bounds for Bayesian linear regression and Dirichlet process mixture models.
A key limitation of sampling algorithms for approximate inference is that it is difficult to quantify their approximation error. Widely used sampling schemes, such as sequential importance sampling with resampling and Metropolis-Hastings, produce output samples drawn from a distribution that may be far from the target posterior distribution. This paper shows how to upper-bound the symmetric KL divergence between the output distribution of a broad class of sequential Monte Carlo (SMC) samplers and their target posterior distributions, subject to assumptions about the accuracy of a separate gold-standard sampler. The proposed method applies to samplers that combine multiple particles, multinomial resampling, and rejuvenation kernels. The experiments show the technique being used to estimate bounds on the divergence of SMC samplers for posterior inference in a Bayesian linear regression model and a Dirichlet process mixture model.