Exact Langevin Dynamics with Stochastic Gradients
This work provides a method to correct the bias in stochastic gradient MCMC algorithms, which is a significant problem for researchers and practitioners relying on these approximate inference techniques.
This paper addresses the bias in stochastic gradient Markov Chain Monte Carlo (SG-MCMC) algorithms, showing that many existing methods cannot be corrected with Metropolis-Hastings due to zero acceptance probabilities. The authors propose using a sampler with realizable backwards trajectories, specifically Gradient-Guided Monte Carlo, which allows for non-zero and computable acceptance probabilities with stochastic gradients.
Stochastic gradient Markov Chain Monte Carlo algorithms are popular samplers for approximate inference, but they are generally biased. We show that many recent versions of these methods (e.g. Chen et al. (2014)) cannot be corrected using Metropolis-Hastings rejection sampling, because their acceptance probability is always zero. We can fix this by employing a sampler with realizable backwards trajectories, such as Gradient-Guided Monte Carlo (Horowitz, 1991), which generalizes stochastic gradient Langevin dynamics (Welling and Teh, 2011) and Hamiltonian Monte Carlo. We show that this sampler can be used with stochastic gradients, yielding nonzero acceptance probabilities, which can be computed even across multiple steps.