Efficient Marginalization-based MCMC Methods for Hierarchical Bayesian Inverse Problems
For practitioners solving large-scale inverse problems (e.g., imaging, heat equation), this work offers computationally efficient MCMC methods that scale to high-dimensional state spaces.
This paper addresses the computational challenges of Gibbs sampling for hierarchical Bayesian inverse problems with large state dimensions. By combining low-rank techniques with marginalization, the proposed delayed acceptance and pseudo-marginalization methods achieve up to 10x speedup in sampling while maintaining statistical efficiency.
Hierarchical models in Bayesian inverse problems are characterized by an assumed prior probability distribution for the unknown state and measurement error precision, and hyper-priors for the prior parameters. Combining these probability models using Bayes' law often yields a posterior distribution that cannot be sampled from directly, even for a linear model with Gaussian measurement error and Gaussian prior. Gibbs sampling can be used to sample from the posterior, but problems arise when the dimension of the state is large. This is because the Gaussian sample required for each iteration can be prohibitively expensive to compute, and because the statistical efficiency of the Markov chain degrades as the dimension of the state increases. The latter problem can be mitigated using marginalization-based techniques, but these can be computationally prohibitive as well. In this paper, we combine the low-rank techniques of Brown, Saibaba, and Vallelian (2018) with the marginalization approach of Rue and Held (2005). We consider two variants of this approach: delayed acceptance and pseudo-marginalization. We provide a detailed analysis of the acceptance rates and computational costs associated with our proposed algorithms, and compare their performances on two numerical test cases---image deblurring and inverse heat equation.