Laplace Matching for fast Approximate Inference in Latent Gaussian Models
This work addresses the computational bottleneck in Bayesian inference for latent Gaussian models, offering a practical solution for researchers and practitioners dealing with non-Gaussian data, though it is incremental as it builds on existing Laplace approximations.
The authors tackled the problem of computationally expensive Bayesian inference on non-Gaussian data by proposing Laplace Matching, a fast approximate inference framework that achieves high approximation quality with minimal computational cost, as demonstrated by experiments showing comparable results to state-of-the-art methods at drastically reduced runtime (e.g., <1 second).
Bayesian inference on non-Gaussian data is often non-analytic and requires computationally expensive approximations such as sampling or variational inference. We propose an approximate inference framework primarily designed to be computationally cheap while still achieving high approximation quality. The concept, which we call Laplace Matching, involves closed-form, approximate, bi-directional transformations between the parameter spaces of exponential families. These are constructed from Laplace approximations under custom-designed basis transformations. The mappings can then be leveraged to effectively turn a latent Gaussian distribution into an approximate conjugate prior to a rich class of observable variables. This allows us to train latent Gaussian models such as Gaussian Processes on non-Gaussian data at nearly no additional cost. The method can be thought of as a pre-processing step which can be implemented in <5 lines of code and runs in less than a second. Furthermore, Laplace Matching yields a simple way to group similar data points together, e.g. to produce inducing points for GPs. We empirically evaluate the method with experiments for four different exponential distributions, namely the Beta, Gamma, Dirichlet and inverse Wishart, showing approximation quality comparable to state-of-the-art approximate inference techniques at a drastic reduction in computational cost.