Minimax Estimation of Quadratic Fourier Functionals
This provides a theoretical foundation for comparing distributions in machine learning, but it is incremental as it extends existing work to new kernel-induced inner products.
The paper tackles the problem of estimating inner products between two nonparametric probability distributions from IID samples, including classical and kernel-induced cases, by proposing estimators and proving they achieve minimax optimal rates with non-asymptotic error bounds.
We study estimation of (semi-)inner products between two nonparametric probability distributions, given IID samples from each distribution. These products include relatively well-studied classical $\mathcal{L}^2$ and Sobolev inner products, as well as those induced by translation-invariant reproducing kernels, for which we believe our results are the first. We first propose estimators for these quantities, and the induced (semi)norms and (pseudo)metrics. We then prove non-asymptotic upper bounds on their mean squared error, in terms of weights both of the inner product and of the two distributions, in the Fourier basis. Finally, we prove minimax lower bounds that imply rate-optimality of the proposed estimators over Fourier ellipsoids.