Fast post-process Bayesian inference with Variational Sparse Bayesian Quadrature
This addresses the need for efficient inference in applied Bayesian scenarios where pre-existing data is available, representing an incremental improvement by leveraging existing evaluations.
The paper tackles the problem of reusing existing model evaluations from optimization runs for fast Bayesian inference, introducing Variational Sparse Bayesian Quadrature (VSBQ) to build high-quality posterior approximations without additional model calls, validated on synthetic and real-world applications.
In applied Bayesian inference scenarios, users may have access to a large number of pre-existing model evaluations, for example from maximum-a-posteriori (MAP) optimization runs. However, traditional approximate inference techniques make little to no use of this available information. We propose the framework of post-process Bayesian inference as a means to obtain a quick posterior approximation from existing target density evaluations, with no further model calls. Within this framework, we introduce Variational Sparse Bayesian Quadrature (VSBQ), a method for post-process approximate inference for models with black-box and potentially noisy likelihoods. VSBQ reuses existing target density evaluations to build a sparse Gaussian process (GP) surrogate model of the log posterior density function. Subsequently, we leverage sparse-GP Bayesian quadrature combined with variational inference to achieve fast approximate posterior inference over the surrogate. We validate our method on challenging synthetic scenarios and real-world applications from computational neuroscience. The experiments show that VSBQ builds high-quality posterior approximations by post-processing existing optimization traces, with no further model evaluations.