Geometric Gaussian Approximations of Probability Distributions
This provides a theoretical foundation for approximation methods in statistics and machine learning, but it is incremental as it builds on existing geometric concepts.
The paper tackles the problem of approximating complex probability distributions, such as Bayesian posteriors, by proving that geometric Gaussian approximations using diffeomorphisms or Riemannian exponential maps are universal and can capture any distribution.
Approximating complex probability distributions, such as Bayesian posterior distributions, is of central interest in many applications. We study the expressivity of geometric Gaussian approximations. These consist of approximations by Gaussian pushforwards through diffeomorphisms or Riemannian exponential maps. We first review these two different kinds of geometric Gaussian approximations. Then we explore their relationship to one another. We further provide a constructive proof that such geometric Gaussian approximations are universal, in that they can capture any probability distribution. Finally, we discuss whether, given a family of probability distributions, a common diffeomorphism can be found to obtain uniformly high-quality geometric Gaussian approximations for that family.