Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptions
This work offers improved theoretical bounds for generative modeling, which is incremental as it refines existing analysis under less restrictive assumptions.
The paper tackles the problem of providing efficient convergence guarantees for score-based generative modeling under minimal assumptions, showing that approximating data distributions in reverse KL divergence with ε-accuracy can be achieved in Õ(d log(1/δ)/ε) steps under conditions like finite second moments or smooth score functions.
We give an improved theoretical analysis of score-based generative modeling. Under a score estimate with small $L^2$ error (averaged across timesteps), we provide efficient convergence guarantees for any data distribution with second-order moment, by either employing early stopping or assuming smoothness condition on the score function of the data distribution. Our result does not rely on any log-concavity or functional inequality assumption and has a logarithmic dependence on the smoothness. In particular, we show that under only a finite second moment condition, approximating the following in reverse KL divergence in $ε$-accuracy can be done in $\tilde O\left(\frac{d \log (1/δ)}ε\right)$ steps: 1) the variance-$δ$ Gaussian perturbation of any data distribution; 2) data distributions with $1/δ$-smooth score functions. Our analysis also provides a quantitative comparison between different discrete approximations and may guide the choice of discretization points in practice.