Non Gaussian Denoising Diffusion Models
This work addresses a specific bottleneck in diffusion models for generative tasks, representing an incremental improvement.
The authors tackled the problem of limited noise distribution flexibility in diffusion models by exploring non-Gaussian noise distributions, showing that Gamma distribution and Gaussian mixtures improve performance for image and speech generation.
Generative diffusion processes are an emerging and effective tool for image and speech generation. In the existing methods, the underline noise distribution of the diffusion process is Gaussian noise. However, fitting distributions with more degrees of freedom, could help the performance of such generative models. In this work, we investigate other types of noise distribution for the diffusion process. Specifically, we show that noise from Gamma distribution provides improved results for image and speech generation. Moreover, we show that using a mixture of Gaussian noise variables in the diffusion process improves the performance over a diffusion process that is based on a single distribution. Our approach preserves the ability to efficiently sample state in the training diffusion process while using Gamma noise and a mixture of noise.