Diffusion Model Conditioning on Gaussian Mixture Model and Negative Gaussian Mixture Gradient
This work addresses a specific issue in generative modeling for researchers, offering incremental improvements in conditioning and training stability for diffusion models.
The paper tackled the problem of improving diffusion model conditioning by using Gaussian mixture models (GMMs) as feature conditioning, showing that this approach reduces defect generations compared to class conditioning, with experiments supporting the findings. It also introduced a novel negative Gaussian mixture gradient (NGMG) function to enhance training stability and theoretically linked it to benefits similar to the Earth Mover distance.
Diffusion models (DMs) are a type of generative model that has a huge impact on image synthesis and beyond. They achieve state-of-the-art generation results in various generative tasks. A great diversity of conditioning inputs, such as text or bounding boxes, are accessible to control the generation. In this work, we propose a conditioning mechanism utilizing Gaussian mixture models (GMMs) as feature conditioning to guide the denoising process. Based on set theory, we provide a comprehensive theoretical analysis that shows that conditional latent distribution based on features and classes is significantly different, so that conditional latent distribution on features produces fewer defect generations than conditioning on classes. Two diffusion models conditioned on the Gaussian mixture model are trained separately for comparison. Experiments support our findings. A novel gradient function called the negative Gaussian mixture gradient (NGMG) is proposed and applied in diffusion model training with an additional classifier. Training stability has improved. We also theoretically prove that NGMG shares the same benefit as the Earth Mover distance (Wasserstein) as a more sensible cost function when learning distributions supported by low-dimensional manifolds.