Variance reduction for additive functional of Markov chains via martingale representations
This work addresses variance reduction for Markov chain simulations, which is crucial for improving efficiency in statistical and machine learning applications, though it appears incremental as it builds on existing martingale and MCMC frameworks.
The authors tackled the problem of reducing variance in additive functionals of Markov chains by developing a non-asymptotic method based on a discrete-time martingale representation, which does not require knowledge of the stationary distribution or specific density structures, and demonstrated that it achieves a smaller cost-to-variance product than naive algorithms, with numerical validation for Langevin-type MCMC methods.
In this paper we propose an efficient variance reduction approach for additive functionals of Markov chains relying on a novel discrete time martingale representation. Our approach is fully non-asymptotic and does not require the knowledge of the stationary distribution (and even any type of ergodicity) or specific structure of the underlying density. By rigorously analyzing the convergence properties of the proposed algorithm, we show that its cost-to-variance product is indeed smaller than one of the naive algorithm. The numerical performance of the new method is illustrated for the Langevin-type Markov Chain Monte Carlo (MCMC) methods.