Cyclical Kernel Adaptive Metropolis
This addresses a specific issue in MCMC sampling for multimodal posteriors, but appears incremental as it modifies existing adaptive methods.
The paper tackles the problem of adaptive Metropolis algorithms failing to converge on bimodal distributions due to being trapped in local modes, and shows that cKAM enables escape from local modes while maintaining high performance.
We propose cKAM, cyclical Kernel Adaptive Metropolis, which incorporates a cyclical stepsize scheme to allow control for exploration and sampling. We show that on a crafted bimodal distribution, existing Adaptive Metropolis type algorithms would fail to converge to the true posterior distribution. We point out that this is because adaptive samplers estimates the local/global covariance structure using past history of the chain, which will lead to adaptive algorithms be trapped in a local mode. We demonstrate that cKAM encourages exploration of the posterior distribution and allows the sampler to escape from a local mode, while maintaining the high performance of adaptive methods.