Score-based sampling without diffusions: Guidance from a simple and modular scheme
This work addresses sampling challenges in machine learning and statistics by providing a simpler, more flexible approach that can leverage existing efficient algorithms, though it is incremental as it builds on prior score-based diffusion methods.
The paper tackles the problem of score-based sampling by introducing a modular scheme that reduces it to solving a sequence of strongly log-concave sampling problems, enabling the use of high-accuracy samplers to achieve ε-accurate results with polynomial dependence on log(1/ε) and √d dimension scaling.
Sampling based on score diffusions has led to striking empirical results, and has attracted considerable attention from various research communities. It depends on availability of (approximate) Stein score functions for various levels of additive noise. We describe and analyze a modular scheme that reduces score-based sampling to solving a short sequence of ``nice'' sampling problems, for which high-accuracy samplers are known. We show how to design forward trajectories such that both (a) the terminal distribution, and (b) each of the backward conditional distribution is defined by a strongly log concave (SLC) distribution. This modular reduction allows us to exploit \emph{any} SLC sampling algorithm in order to traverse the backwards path, and we establish novel guarantees with short proofs for both uni-modal and multi-modal densities. The use of high-accuracy routines yields $\varepsilon$-accurate answers, in either KL or Wasserstein distances, with polynomial dependence on $\log(1/\varepsilon)$ and $\sqrt{d}$ dependence on the dimension.