Diffuse and Disperse: Image Generation with Representation Regularization
This work addresses a gap in generative modeling for image generation, offering a simple, incremental improvement to diffusion models.
The authors tackled the lack of explicit regularization in diffusion-based generative models by proposing Dispersive Loss, a plug-and-play regularizer that improves image generation on ImageNet, reporting consistent improvements over strong baselines.
The development of diffusion-based generative models over the past decade has largely proceeded independently of progress in representation learning. These diffusion models typically rely on regression-based objectives and generally lack explicit regularization. In this work, we propose \textit{Dispersive Loss}, a simple plug-and-play regularizer that effectively improves diffusion-based generative models. Our loss function encourages internal representations to disperse in the hidden space, analogous to contrastive self-supervised learning, with the key distinction that it requires no positive sample pairs and therefore does not interfere with the sampling process used for regression. Compared to the recent method of representation alignment (REPA), our approach is self-contained and minimalist, requiring no pre-training, no additional parameters, and no external data. We evaluate Dispersive Loss on the ImageNet dataset across a range of models and report consistent improvements over widely used and strong baselines. We hope our work will help bridge the gap between generative modeling and representation learning.