Ensemble Slice Sampling: Parallel, black-box and gradient-free inference for correlated & multimodal distributions
This provides a parallel, black-box, and gradient-free inference method ideal for scientific fields like physics and cosmology, though it is an incremental improvement over existing MCMC techniques.
The paper tackled the sensitivity of Slice Sampling to hyperparameters and its struggles with correlated or multimodal distributions by introducing Ensemble Slice Sampling, which improved efficiency by over an order of magnitude on highly correlated targets and handled multimodal distributions in high dimensions.
Slice Sampling has emerged as a powerful Markov Chain Monte Carlo algorithm that adapts to the characteristics of the target distribution with minimal hand-tuning. However, Slice Sampling's performance is highly sensitive to the user-specified initial length scale hyperparameter and the method generally struggles with poorly scaled or strongly correlated distributions. This paper introduces Ensemble Slice Sampling (ESS), a new class of algorithms that bypasses such difficulties by adaptively tuning the initial length scale and utilising an ensemble of parallel walkers in order to efficiently handle strong correlations between parameters. These affine-invariant algorithms are trivial to construct, require no hand-tuning, and can easily be implemented in parallel computing environments. Empirical tests show that Ensemble Slice Sampling can improve efficiency by more than an order of magnitude compared to conventional MCMC methods on a broad range of highly correlated target distributions. In cases of strongly multimodal target distributions, Ensemble Slice Sampling can sample efficiently even in high dimensions. We argue that the parallel, black-box and gradient-free nature of the method renders it ideal for use in scientific fields such as physics, astrophysics and cosmology which are dominated by a wide variety of computationally expensive and non-differentiable models.