Non-Uniform Diffusion Models
This work addresses the computational bottleneck in diffusion models for generative modeling, offering a speed-up that is particularly beneficial for high-resolution image generation.
The paper tackles the inefficiency of standard diffusion models by introducing non-uniform diffusion, which leads to multi-scale models that achieve better FID scores in similar or less training time and generate samples 4.4 times faster at 128x128 resolution.
Diffusion models have emerged as one of the most promising frameworks for deep generative modeling. In this work, we explore the potential of non-uniform diffusion models. We show that non-uniform diffusion leads to multi-scale diffusion models which have similar structure to this of multi-scale normalizing flows. We experimentally find that in the same or less training time, the multi-scale diffusion model achieves better FID score than the standard uniform diffusion model. More importantly, it generates samples $4.4$ times faster in $128\times 128$ resolution. The speed-up is expected to be higher in higher resolutions where more scales are used. Moreover, we show that non-uniform diffusion leads to a novel estimator for the conditional score function which achieves on par performance with the state-of-the-art conditional denoising estimator. Our theoretical and experimental findings are accompanied by an open source library MSDiff which can facilitate further research of non-uniform diffusion models.