Geometric convergence of elliptical slice sampling
This work offers convergence guarantees for a widely used, tuning-free sampling method in Bayesian learning, though it is incremental as it builds on existing theory.
The paper proves that the elliptical slice sampler, a Markov chain method for Bayesian posterior sampling, is geometrically ergodic under weak regularity assumptions, providing qualitative convergence guarantees. Numerical experiments show dimension-independent performance even in cases not covered by the theoretical result.
For Bayesian learning, given likelihood function and Gaussian prior, the elliptical slice sampler, introduced by Murray, Adams and MacKay 2010, provides a tool for the construction of a Markov chain for approximate sampling of the underlying posterior distribution. Besides of its wide applicability and simplicity its main feature is that no tuning is necessary. Under weak regularity assumptions on the posterior density we show that the corresponding Markov chain is geometrically ergodic and therefore yield qualitative convergence guarantees. We illustrate our result for Gaussian posteriors as they appear in Gaussian process regression, as well as in a setting of a multi-modal distribution. Remarkably, our numerical experiments indicate a dimension-independent performance of elliptical slice sampling even in situations where our ergodicity result does not apply.