Likelihood-informed dimension reduction across tempered Bayesian posteriors
For practitioners of Bayesian inversion with expensive simulators, this work provides a more robust dimension reduction technique that works under severe data limitations and noisy gradients.
The paper generalizes likelihood-informed dimension reduction to tempered Bayesian posteriors, enabling robust dimension reduction for inverse problems with limited and noisy data. The proposed α-LIS method with accumulated gradients outperforms the theoretically optimal α=1 in challenging scenarios.
Scientific computer simulations cannot represent all scales in realistic applications. To bridge this model-data gap, parameters are injected into models and constrained with noisy data using Bayesian inversion. To reduce the number of simulator evaluations, which can be 10^5 or more, modern approaches employ dimension reduction in conjunction with emulation of the forward map (that contains the simulator). Due to scarcity of model evaluations and data, this dimension reduction becomes very important for posterior sampling performance. Recent work on likelihood-informed subspaces (LIS) truncates to informative directions by optimizing bounds on information loss, and though mathematically well-adapted to sampling, they are often restrictive in practice. In this work, we provably generalize this methodology to facilitate application to $α$-tempered (i.e., annealed, power-posterior) distributions for $α$ in [0,1]. We provide theory to build partially-informed spaces termed $α$-LIS. We show how $α$ < 1 can often produce near-optimal spaces. In addition, we focus on applying $α$-LIS to practical cases, where the available data is severely limited and noisy. We propose and test extensions for utilizing data from the entire sequence of distributions $α$_0 < ... < $α$_k, and use simple approximations of model gradients so that our approach can be used for emulation of forward maps for chaotic or stochastic systems where derivatives are unavailable or uninformative due to noise. In experiments, our accumulated approach is much more robust to these challenging circumstances than the theoretically optimal $α$ = 1.