Scalable Sobolev IPM for Probability Measures on a Graph
It solves a long-standing computational bottleneck for researchers and practitioners using Sobolev IPM in machine learning, though it is incremental as it builds on existing IPM theory.
The paper tackles the computational inefficiency of Sobolev IPM for probability measures on graphs by introducing a novel regularization that yields a closed-form expression, enabling fast computation and practical applications in large-scale settings, with preliminary evidence showing advantages in document classification and topological data analysis.
We investigate the Sobolev IPM problem for probability measures supported on a graph metric space. Sobolev IPM is an important instance of integral probability metrics (IPM), and is obtained by constraining a critic function within a unit ball defined by the Sobolev norm. In particular, it has been used to compare probability measures and is crucial for several theoretical works in machine learning. However, to our knowledge, there are no efficient algorithmic approaches to compute Sobolev IPM effectively, which hinders its practical applications. In this work, we establish a relation between Sobolev norm and weighted $L^p$-norm, and leverage it to propose a \emph{novel regularization} for Sobolev IPM. By exploiting the graph structure, we demonstrate that the regularized Sobolev IPM provides a \emph{closed-form} expression for fast computation. This advancement addresses long-standing computational challenges, and paves the way to apply Sobolev IPM for practical applications, even in large-scale settings. Additionally, the regularized Sobolev IPM is negative definite. Utilizing this property, we design positive-definite kernels upon the regularized Sobolev IPM, and provide preliminary evidences of their advantages for comparing probability measures on a given graph for document classification and topological data analysis.