Semi-Separable Hamiltonian Monte Carlo for Inference in Bayesian Hierarchical Models
This addresses the problem of computational inefficiency in MCMC sampling for statisticians and data scientists, representing an incremental improvement over existing RMHMC techniques.
The paper tackles the challenge of sampling from hierarchical Bayesian models by introducing semi-separable Hamiltonian Monte Carlo, a new Riemannian manifold HMC method that uses a specialized mass matrix and integrator to achieve faster mixing than Gibbs sampling while being more efficient than previous RMHMC methods.
Sampling from hierarchical Bayesian models is often difficult for MCMC methods, because of the strong correlations between the model parameters and the hyperparameters. Recent Riemannian manifold Hamiltonian Monte Carlo (RMHMC) methods have significant potential advantages in this setting, but are computationally expensive. We introduce a new RMHMC method, which we call semi-separable Hamiltonian Monte Carlo, which uses a specially designed mass matrix that allows the joint Hamiltonian over model parameters and hyperparameters to decompose into two simpler Hamiltonians. This structure is exploited by a new integrator which we call the alternating blockwise leapfrog algorithm. The resulting method can mix faster than simpler Gibbs sampling while being simpler and more efficient than previous instances of RMHMC.