Asymptotic bias of inexact Markov Chain Monte Carlo methods in high dimension
This work addresses the theoretical understanding of bias in approximate MCMC methods for high-dimensional statistical inference, which is incremental but provides precise bounds for specific model classes.
The paper analyzes the asymptotic bias of inexact Markov Chain Monte Carlo methods like ULA and uHMC in high dimensions, establishing bounds on Wasserstein distances between invariant measures and target distributions, with results showing similar dependence on step size and dimension for models like mean-field and finite-range graphical models.
Inexact Markov Chain Monte Carlo methods rely on Markov chains that do not exactly preserve the target distribution. Examples include the unadjusted Langevin algorithm (ULA) and unadjusted Hamiltonian Monte Carlo (uHMC). This paper establishes bounds on Wasserstein distances between the invariant probability measures of inexact MCMC methods and their target distributions with a focus on understanding the precise dependence of this asymptotic bias on both dimension and discretization step size. Assuming Wasserstein bounds on the convergence to equilibrium of either the exact or the approximate dynamics, we show that for both ULA and uHMC, the asymptotic bias depends on key quantities related to the target distribution or the stationary probability measure of the scheme. As a corollary, we conclude that for models with a limited amount of interactions such as mean-field models, finite range graphical models, and perturbations thereof, the asymptotic bias has a similar dependence on the step size and the dimension as for product measures.