A Markov Jump Process for More Efficient Hamiltonian Monte Carlo
This work addresses a bottleneck in sampling efficiency for practitioners in statistics and machine learning, though it appears incremental as it modifies an existing method.
The authors tackled the constraint of transition rates being limited to ≤1 in sampling algorithms like Hamiltonian Monte Carlo by deriving a continuous-time Markov jump process version, which improved mixing in example problems as shown by spectral gap and autocorrelation metrics.
In most sampling algorithms, including Hamiltonian Monte Carlo, transition rates between states correspond to the probability of making a transition in a single time step, and are constrained to be less than or equal to 1. We derive a Hamiltonian Monte Carlo algorithm using a continuous time Markov jump process, and are thus able to escape this constraint. Transition rates in a Markov jump process need only be non-negative. We demonstrate that the new algorithm leads to improved mixing for several example problems, both by evaluating the spectral gap of the Markov operator, and by computing autocorrelation as a function of compute time. We release the algorithm as an open source Python package.