Condition-number-independent convergence rate of Riemannian Hamiltonian Monte Carlo with numerical integrators
This provides theoretical guarantees for efficient sampling in high-dimensional, ill-conditioned settings, complementing empirical results, but it is incremental as it builds on existing methods.
The paper tackles the problem of sampling from constrained distributions using Riemannian Hamiltonian Monte Carlo, showing that the convergence rate of certain numerical integrators is independent of condition number and geometry, with a mixing time of O~(mn^3) to achieve ε total variation distance for specific distributions.
We study the convergence rate of discretized Riemannian Hamiltonian Monte Carlo on sampling from distributions in the form of $e^{-f(x)}$ on a convex body $\mathcal{M}\subset\mathbb{R}^{n}$. We show that for distributions in the form of $e^{-α^{\top}x}$ on a polytope with $m$ constraints, the convergence rate of a family of commonly-used integrators is independent of $\left\Vert α\right\Vert _{2}$ and the geometry of the polytope. In particular, the implicit midpoint method (IMM) and the generalized Leapfrog method (LM) have a mixing time of $\widetilde{O}\left(mn^{3}\right)$ to achieve $ε$ total variation distance to the target distribution. These guarantees are based on a general bound on the convergence rate for densities of the form $e^{-f(x)}$ in terms of parameters of the manifold and the integrator. Our theoretical guarantee complements the empirical results of [KLSV22], which shows that RHMC with IMM can sample ill-conditioned, non-smooth and constrained distributions in very high dimension efficiently in practice.