Efficient Learning of Stationary Diffusions with Stein-type Discrepancies
This provides a more efficient method for learning stationary diffusions, which is important for applications in generative modeling and sampling, though it appears incremental as it builds on existing kernel-based approaches.
The paper tackles the problem of learning stationary diffusions by introducing Stein-type KDS (SKDS), which guarantees alignment between the learned diffusion's stationary distribution and the target distribution. Empirically, SKDS achieves comparable accuracy to existing methods while substantially reducing computational cost and outperforming most competitive baselines.
Learning a stationary diffusion amounts to estimating the parameters of a stochastic differential equation whose stationary distribution matches a target distribution. We build on the recently introduced kernel deviation from stationarity (KDS), which enforces stationarity by evaluating expectations of the diffusion's generator in a reproducing kernel Hilbert space. Leveraging the connection between KDS and Stein discrepancies, we introduce the Stein-type KDS (SKDS) as an alternative formulation. We prove that a vanishing SKDS guarantees alignment of the learned diffusion's stationary distribution with the target. Furthermore, under broad parametrizations, SKDS is convex with an empirical version that is $ε$-quasiconvex with high probability. Empirically, learning with SKDS attains comparable accuracy to KDS while substantially reducing computational cost and yields improvements over the majority of competitive baselines.