Correlated pseudo-marginal Metropolis-Hastings using quasi-Newton proposals
This is an incremental improvement for practitioners in computational statistics and Bayesian inference dealing with high-dimensional sampling challenges.
The authors tackled the problem of inefficient proposal distributions in pseudo-marginal Metropolis-Hastings (pmMH) for high-dimensional target sampling by extending it to use quasi-Newton methods based on past iterations, resulting in improved performance over Hessian-based proposals in several problems.
Pseudo-marginal Metropolis-Hastings (pmMH) is a versatile algorithm for sampling from target distributions which are not easy to evaluate point-wise. However, pmMH requires good proposal distributions to sample efficiently from the target, which can be problematic to construct in practice. This is especially a problem for high-dimensional targets when the standard random-walk proposal is inefficient. We extend pmMH to allow for constructing the proposal based on information from multiple past iterations. As a consequence, quasi-Newton (qN) methods can be employed to form proposals which utilize gradient information to guide the Markov chain to areas of high probability and to construct approximations of the local curvature to scale step sizes. The proposed method is demonstrated on several problems which indicate that qN proposals can perform better than other common Hessian-based proposals.