On the Geometric Ergodicity of Hamiltonian Monte Carlo
This addresses theoretical convergence guarantees for HMC, a widely used method in Bayesian statistics and machine learning, providing guidelines for parameter tuning.
The paper establishes conditions for when Hamiltonian Monte Carlo (HMC) Markov chains are or are not geometrically ergodic, finding that with position-independent integration times, ergodicity requires a gradient pointing towards the center and growing linearly, while position-dependent times can recover ergodicity for broader tail behaviors.
We establish general conditions under which Markov chains produced by the Hamiltonian Monte Carlo method will and will not be geometrically ergodic. We consider implementations with both position-independent and position-dependent integration times. In the former case we find that the conditions for geometric ergodicity are essentially a gradient of the log-density which asymptotically points towards the centre of the space and grows no faster than linearly. In an idealised scenario in which the integration time is allowed to change in different regions of the space, we show that geometric ergodicity can be recovered for a much broader class of tail behaviours, leading to some guidelines for the choice of this free parameter in practice.