Structured Diffusion Models with Mixture of Gaussians as Prior Distribution
This work addresses the need for more efficient and robust diffusion models in machine learning, though it appears incremental as it builds on existing diffusion frameworks.
The authors tackled the problem of improving diffusion models by using a mixture of Gaussians as the prior distribution to incorporate structured data information, resulting in a model that is robust to mis-specifications and suitable for limited training resources or faster real-time training.
We propose a class of structured diffusion models, in which the prior distribution is chosen as a mixture of Gaussians, rather than a standard Gaussian distribution. The specific mixed Gaussian distribution, as prior, can be chosen to incorporate certain structured information of the data. We develop a simple-to-implement training procedure that smoothly accommodates the use of mixed Gaussian as prior. Theory is provided to quantify the benefits of our proposed models, compared to the classical diffusion models. Numerical experiments with synthetic, image and operational data are conducted to show comparative advantages of our model. Our method is shown to be robust to mis-specifications and in particular suits situations where training resources are limited or faster training in real time is desired.