Schoenberg-Rao distances: Entropy-based and geometry-aware statistical Hilbert distances
This provides a geometry-aware alternative to Wasserstein distances for machine learning practitioners, though it is incremental as it builds on existing statistical distances.
The paper introduces Schoenberg-Rao distances, a generalization of Maximum Mean Discrepancy that uses conditionally negative semi-definite kernels to compare probability distributions, with closed-form expressions for Gaussian mixtures, and demonstrates efficiency in tasks like density estimation and generative modeling.
Distances between probability distributions that take into account the geometry of their sample space,like the Wasserstein or the Maximum Mean Discrepancy (MMD) distances have received a lot of attention in machine learning as they can, for instance, be used to compare probability distributions with disjoint supports. In this paper, we study a class of statistical Hilbert distances that we term the Schoenberg-Rao distances, a generalization of the MMD that allows one to consider a broader class of kernels, namely the conditionally negative semi-definite kernels. In particular, we introduce a principled way to construct such kernels and derive novel closed-form distances between mixtures of Gaussian distributions. These distances, derived from the concave Rao's quadratic entropy, enjoy nice theoretical properties and possess interpretable hyperparameters which can be tuned for specific applications. Our method constitutes a practical alternative to Wasserstein distances and we illustrate its efficiency on a broad range of machine learning tasks such as density estimation, generative modeling and mixture simplification.