Policy Gradients for Optimal Parallel Tempering MCMC
This addresses a bottleneck in MCMC sampling for complex distributions, though it appears incremental as it builds on known adaptive methods.
The paper tackles the problem of selecting optimal temperatures for parallel tempering MCMC to improve mixing in multi-modal distributions, and the result is an adaptive algorithm using policy gradients that achieves lower integrated autocorrelation times compared to existing schemes on benchmarks.
Parallel tempering is a meta-algorithm for Markov Chain Monte Carlo that uses multiple chains to sample from tempered versions of the target distribution, enhancing mixing in multi-modal distributions that are challenging for traditional methods. The effectiveness of parallel tempering is heavily influenced by the selection of chain temperatures. Here, we present an adaptive temperature selection algorithm that dynamically adjusts temperatures during sampling using a policy gradient approach. Experiments demonstrate that our method can achieve lower integrated autocorrelation times compared to traditional geometrically spaced temperatures and uniform acceptance rate schemes on benchmark distributions.