Cloud Diffusion Part 1: Theory and Motivation
This addresses a theoretical limitation in diffusion models for image generation, potentially improving efficiency and quality for AI applications in visual content creation.
The paper tackles the mismatch between white noise used in diffusion models and the scale-invariant properties of natural images, proposing to replace white noise with scale-invariant noise profiles to form Cloud Diffusion Models. It argues this can lead to faster inference, improved high-frequency details, and greater controllability, with a follow-up planned for empirical validation.
Diffusion models for image generation function by progressively adding noise to an image set and training a model to separate out the signal from the noise. The noise profile used by these models is white noise -- that is, noise based on independent normal distributions at each point whose mean and variance is independent of the scale. By contrast, most natural image sets exhibit a type of scale invariance in their low-order statistical properties characterized by a power-law scaling. Consequently, natural images are closer (in a quantifiable sense) to a different probability distribution that emphasizes large scale correlations and de-emphasizes small scale correlations. These scale invariant noise profiles can be incorporated into diffusion models in place of white noise to form what we will call a ``Cloud Diffusion Model". We argue that these models can lead to faster inference, improved high-frequency details, and greater controllability. In a follow-up paper, we will build and train a Cloud Diffusion Model that uses scale invariance at a fundamental level and compare it to classic, white noise diffusion models.