SVGD as a kernelized Wasserstein gradient flow of the chi-squared divergence
This work addresses sampling algorithms for machine learning by offering a new theoretical perspective and improved method, though it is incremental in building on SVGD.
The paper reinterprets Stein Variational Gradient Descent (SVGD) as a kernelized gradient flow of the chi-squared divergence, demonstrating uniform exponential ergodicity under weak conditions, and proposes Laplacian Adjusted Wasserstein Gradient Descent (LAWGD) as an alternative with strong convergence guarantees and good practical performance.
Stein Variational Gradient Descent (SVGD), a popular sampling algorithm, is often described as the kernelized gradient flow for the Kullback-Leibler divergence in the geometry of optimal transport. We introduce a new perspective on SVGD that instead views SVGD as the (kernelized) gradient flow of the chi-squared divergence which, we show, exhibits a strong form of uniform exponential ergodicity under conditions as weak as a Poincaré inequality. This perspective leads us to propose an alternative to SVGD, called Laplacian Adjusted Wasserstein Gradient Descent (LAWGD), that can be implemented from the spectral decomposition of the Laplacian operator associated with the target density. We show that LAWGD exhibits strong convergence guarantees and good practical performance.