Covariance-Informed Subspace: an Adaptive Gradient-Free Input Dimension Reduction Method for Bayesian Inference
It provides a dimension reduction method for Bayesian inference when gradients are unavailable, addressing a practical bottleneck in simulation-based inference.
The paper proposes a gradient-free dimension reduction method for Bayesian inference in high-dimensional fields, using a posterior-to-prior covariance ratio indicator. It achieves better posterior approximation in linear Gaussian cases and demonstrates effectiveness in groundwater and atmospheric applications.
This paper addresses the challenge of dimension reduction (DR) in Bayesian inference of high-resolution two-or three-dimensional fields, where a priori parametrizations require a large number of terms. The underlying idea is common to state-of-the-art methods in which the parameter space is decomposed into two subspaces, one informed by the likelihood and one constrained by the prior. DR techniques generally use gradient information from the log-likelihood to derive the corresponding subspaces. However, the gradient may be unavailable or expensive to compute accurately, for instance in the case of simulation-based inference. Inspired by approaches based on likelihood-informed subspaces, we develop a new DR method tailored for settings where gradient computation is not feasible. More specifically, we propose a gradient-free indicator for determining whether a direction is informed by the data. This indicator is derived from the posterior-to-prior covariance ratio introduced in Spantini et al. (2015). We show that, in the linear Gaussian case, this indicator combined with an approximate likelihood leads to a better posterior approximation. The method is then extended to nonlinear cases, and strategies to approximate the posterior covariance are detailed. We demonstrate the effectiveness of this DR through two high-dimensional inference problems arising from groundwater and atmospheric applications.