Sobolev Regularized MMD Gradient Flow
This work provides a unified gradient flow method for both sampling and generative modeling with theoretical convergence guarantees, addressing a gap in prior works.
The authors propose Sobolev-regularized MMD gradient flow (SrMMD) that achieves provable global convergence guarantees in MMD without isoperimetric assumptions, and is applicable to both sampling and generative modeling. Empirical results demonstrate effectiveness across a broad range of tasks.
We propose Sobolev-regularized Maximum Mean Discrepancy (SrMMD) gradient flow, a regularized variant of maximum mean discrepancy (MMD) gradient flow based on a gradient penalty on the witness function. The proposed regularization mitigates the non-convexity of the MMD objective and yields provable \emph{global} convergence guarantees in MMD in both continuous and discrete time. A more surprising appeal is that our convergence analysis does not rely on isoperimetric assumptions on the target distribution. Instead, it is based on a regularity condition on the difference between kernel mean embeddings. A key highlight of the proposed flow is that it is applicable in both sampling (from an unnormalized target distribution) -- using Stein kernels -- and generative modeling settings, unlike previous works, where a gradient flow is suitable for only generative modeling or sampling but not both. The effectiveness of the proposed flow is empirically verified on a broad range of tasks in both generative modelling and sampling.