Riesz-Kernel Stein Variational Gradient Descent: Renormalized Entropy and Long-Time Particle Limits
This work provides rigorous convergence guarantees for SVGD with singular Riesz kernels, addressing a key theoretical gap for practitioners using such kernels in sampling and inference.
The authors prove a many-particle, long-time sampling theorem for periodic Riesz-kernel Stein variational gradient descent (SVGD) with self-interaction removed, showing that the time-averaged empirical measure converges weakly to the target distribution as particle number and averaging horizon tend to infinity, with explicit algebraic error bounds below the logarithmic singularity threshold.
Stein variational gradient descent (SVGD) transports interacting particles toward a target distribution through deterministic kernelized dynamics. Singular Riesz kernels are attractive because they can provide quantitative population-level convergence, but at the finite-particle level the corresponding Stein energy has infinite self-interaction. We study periodic Riesz SVGD with self-interaction removed and prove a many-particle, long-time sampling theorem. Throughout the range in which the singular Stein energy is locally integrable, under a uniform bound on the initial relative entropy per particle, the time-averaged empirical-measure law converges weakly to the point mass \(δ_π\) at the target as the particle number and any diverging averaging horizon tend to infinity. We also show that the empirical-measure laws induced by invariant particle laws of finite relative entropy converge weakly to \(δ_π\), without a uniform entropy bound. Below the logarithmic singularity threshold, we obtain an explicit algebraic finite-particle error bound. These results extend the joint-entropy approach for smooth-kernel SVGD to singular interactions.