Regularized Finite Dimensional Kernel Sobolev Discrepancy
This work clarifies a theoretical connection in machine learning for researchers in generative models and optimal transport, but it is incremental as it builds on existing discrepancies and approximation methods.
The paper shows that the Sobolev Discrepancy used in generative adversarial networks is equivalent to a weighted negative Sobolev norm, which linearizes the Wasserstein distance and is key in optimal transport theory, and it provides a finite-sample approximation method with error bounds depending on kernel approximation and statistical errors.
We show in this note that the Sobolev Discrepancy introduced in Mroueh et al in the context of generative adversarial networks, is actually the weighted negative Sobolev norm $||.||_{\dot{H}^{-1}(ν_q)}$, that is known to linearize the Wasserstein $W_2$ distance and plays a fundamental role in the dynamic formulation of optimal transport of Benamou and Brenier. Given a Kernel with finite dimensional feature map we show that the Sobolev discrepancy can be approximated from finite samples. Assuming this discrepancy is finite, the error depends on the approximation error in the function space induced by the finite dimensional feature space kernel and on a statistical error due to the finite sample approximation.