Tempering by Subsampling
This work addresses computational efficiency issues in Bayesian inference, particularly for high-dimensional models, though it appears incremental as it builds on existing tempering techniques.
The paper tackles the challenge of improving Markov chain Monte Carlo samplers for Bayesian models by introducing tempering through recursive subsampling without replacement, resulting in increased effective sample size per computation and reduced computational cost compared to traditional tempering methods.
In this paper we demonstrate that tempering Markov chain Monte Carlo samplers for Bayesian models by recursively subsampling observations without replacement can improve the performance of baseline samplers in terms of effective sample size per computation. We present two tempering by subsampling algorithms, subsampled parallel tempering and subsampled tempered transitions. We provide an asymptotic analysis of the computational cost of tempering by subsampling, verify that tempering by subsampling costs less than traditional tempering, and demonstrate both algorithms on Bayesian approaches to learning the mean of a high dimensional multivariate Normal and estimating Gaussian process hyperparameters.