New affine invariant ensemble samplers and their dimensional scaling
This work addresses sampling challenges in computational statistics and machine learning, particularly for high-dimensional inference, but it appears incremental as it builds on existing ensemble and HMC methods.
The authors tackled the problem of sampling from high-dimensional and skewed distributions by introducing new affine invariant ensemble samplers, including a derivative-free version that outperforms existing algorithms like those in the emcee package and a derivative-based HMC version that scales better with dimension.
We introduce new affine invariant ensemble samplers that are easy to construct and improve upon existing algorithms, especially for high-dimensional problems. Specifically, we propose a derivative-free ensemble side move sampler that performs favorably compared to popular samplers in the $\texttt{emcee}$ package. Additionally, we develop a class of derivative-based ensemble Hamiltonian Monte Carlo (HMC) samplers with affine invariance, which outperform standard HMC without affine invariance when sampling highly skewed distributions. We provide asymptotic scaling analysis for high-dimensional Gaussian targets to further elucidate the properties of these affine invariant ensemble samplers. In particular, with derivative information, the affine invariant ensemble HMC can scale much better with dimension compared to derivative-free ensemble samplers.