Towards Unifying Hamiltonian Monte Carlo and Slice Sampling
This provides a theoretical unification for MCMC methods, potentially improving sampling efficiency for Bayesian inference, though it appears incremental as an extension of existing techniques.
The paper unifies Hamiltonian Monte Carlo and slice sampling through the Hamiltonian-Jacobi equation, enabling extension to a broader family of Monomial Gamma Samplers (MGS). It proves that MGS draws decorrelated samples in a parameter limit, with performance gains balanced by numerical difficulty and convergence issues, validated on synthetic and real-world data.
We unify slice sampling and Hamiltonian Monte Carlo (HMC) sampling, demonstrating their connection via the Hamiltonian-Jacobi equation from Hamiltonian mechanics. This insight enables extension of HMC and slice sampling to a broader family of samplers, called Monomial Gamma Samplers (MGS). We provide a theoretical analysis of the mixing performance of such samplers, proving that in the limit of a single parameter, the MGS draws decorrelated samples from the desired target distribution. We further show that as this parameter tends toward this limit, performance gains are achieved at a cost of increasing numerical difficulty and some practical convergence issues. Our theoretical results are validated with synthetic data and real-world applications.