Hess-MC2: Sequential Monte Carlo Squared using Hessian Information and Second Order Proposals
This work addresses computational bottlenecks in Bayesian inference for high-performance computing environments, but it is incremental as it extends existing gradient-based methods with second-order information within a specific framework.
The paper tackled the problem of improving posterior approximation accuracy and computational efficiency in Sequential Monte Carlo Squared (SMC$^2$) methods by incorporating second-order Hessian information into proposal distributions, resulting in enhanced step-size selection and accuracy compared to other proposals in synthetic models.
When performing Bayesian inference using Sequential Monte Carlo (SMC) methods, two considerations arise: the accuracy of the posterior approximation and computational efficiency. To address computational demands, Sequential Monte Carlo Squared (SMC$^2$) is well-suited for high-performance computing (HPC) environments. The design of the proposal distribution within SMC$^2$ can improve accuracy and exploration of the posterior as poor proposals may lead to high variance in importance weights and particle degeneracy. The Metropolis-Adjusted Langevin Algorithm (MALA) uses gradient information so that particles preferentially explore regions of higher probability. In this paper, we extend this idea by incorporating second-order information, specifically the Hessian of the log-target. While second-order proposals have been explored previously in particle Markov Chain Monte Carlo (p-MCMC) methods, we are the first to introduce them within the SMC$^2$ framework. Second-order proposals not only use the gradient (first-order derivative), but also the curvature (second-order derivative) of the target distribution. Experimental results on synthetic models highlight the benefits of our approach in terms of step-size selection and posterior approximation accuracy when compared to other proposals.