Parallel-tempered Stochastic Gradient Hamiltonian Monte Carlo for Approximate Multimodal Posterior Sampling
This addresses the challenge of approximate multimodal posterior sampling in deep Bayesian learning for large-scale applications, representing an incremental improvement by combining existing techniques.
The paper tackles the problem of sampling from complex multimodal posterior distributions with stochastic gradient noise, proposing a new sampler that integrates parallel tempering with Nosé-Hoover dynamics to efficiently draw representative samples, potentially facilitating deep Bayesian learning on large datasets.
We propose a new sampler that integrates the protocol of parallel tempering with the Nosé-Hoover (NH) dynamics. The proposed method can efficiently draw representative samples from complex posterior distributions with multiple isolated modes in the presence of noise arising from stochastic gradient. It potentially facilitates deep Bayesian learning on large datasets where complex multimodal posteriors and mini-batch gradient are encountered.