Score-based deterministic density sampling
This work addresses a fundamental problem in machine learning for efficient and stable sampling, offering a deterministic alternative to stochastic methods with potential applications in generative modeling and Bayesian inference.
The paper tackles the problem of sampling from an unnormalized target density given only its score by proposing a deterministic sampling framework using Score-Based Transport Modeling, which achieves monotone noise-free convergence and produces high-quality generations in as few as 15 steps on high-dimensional image data.
We propose a deterministic sampling framework using Score-Based Transport Modeling for sampling an unnormalized target density $π$ given only its score $\nabla \log π$. Our method approximates the Wasserstein gradient flow on $\mathrm{KL}(f_t\|π)$ by learning the time-varying score $\nabla \log f_t$ on the fly using score matching. While having the same marginal distribution as Langevin dynamics, our method produces smooth deterministic trajectories, resulting in monotone noise-free convergence. We prove that our method dissipates relative entropy at the same rate as the exact gradient flow, provided sufficient training. Numerical experiments validate our theoretical findings: our method converges at the optimal rate, has smooth trajectories, and is often more sample efficient than its stochastic counterpart. Experiments on high-dimensional image data show that our method produces high-quality generations in as few as 15 steps and exhibits natural exploratory behavior. The memory and runtime scale linearly in the sample size.