A Uniform Concentration Inequality for Kernel-Based Two-Sample Statistics
This work addresses the need for performance guarantees in statistical and machine learning methods that rely on distribution metrics, offering theoretical support for applications like dimension reduction and generative models, but it is incremental as it builds on existing kernel-based frameworks.
The paper tackles the problem of optimizing objective functions that depend on discrepancies between probability distributions, such as Energy Distance and Maximum Mean Discrepancy, by establishing a uniform concentration inequality for kernel-based two-sample statistics, providing finite-sample and asymptotic error bounds for estimation errors.
In many contemporary statistical and machine learning methods, one needs to optimize an objective function that depends on the discrepancy between two probability distributions. The discrepancy can be referred to as a metric for distributions. Widely adopted examples of such a metric include Energy Distance (ED), distance Covariance (dCov), Maximum Mean Discrepancy (MMD), and the Hilbert-Schmidt Independence Criterion (HSIC). We show that these metrics can be unified under a general framework of kernel-based two-sample statistics. This paper establishes a novel uniform concentration inequality for the aforementioned kernel-based statistics. Our results provide upper bounds for estimation errors in the associated optimization problems, thereby offering both finite-sample and asymptotic performance guarantees. As illustrative applications, we demonstrate how these bounds facilitate the derivation of error bounds for procedures such as distance covariance-based dimension reduction, distance covariance-based independent component analysis, MMD-based fairness-constrained inference, MMD-based generative model search, and MMD-based generative adversarial networks.