Stable Derivative Free Gaussian Mixture Variational Inference for Bayesian Inverse Problems
This addresses the challenge of stable and efficient Bayesian inference for large-scale inverse problems in scientific computing, though it appears incremental as it builds on existing variational inference techniques.
The paper tackled approximating complex posterior distributions in Bayesian inverse problems with costly forward models and inaccessible gradients, developing a derivative-free variational inference method that demonstrated effectiveness in high-dimensional scenarios and a large-scale Navier-Stokes application.
This paper is concerned with the approximation of probability distributions known up to normalization constants, with a focus on Bayesian inference for large-scale inverse problems in scientific computing. In this context, key challenges include costly repeated evaluations of forward models, multimodality, and inaccessible gradients for the forward model. To address them, we develop a variational inference framework that combines Fisher-Rao natural gradient with specialized quadrature rules to enable derivative free updates of Gaussian mixture variational families. The resulting method, termed Derivative Free Gaussian Mixture Variational Inference (DF-GMVI), guarantees covariance positivity and affine invariance, offering a stable and efficient framework for approximating complex posterior distributions. The effectiveness of DF-GMVI is demonstrated through numerical experiments on challenging scenarios, including distributions with multiple modes, infinitely many modes, and curved modes in spaces with up to 100 dimensions. The method's practicality is further demonstrated in a large-scale application, where it successfully recovers the initial conditions of the Navier-Stokes equations from solution data at positive times.