It Takes Two to Tango: Two Parallel Samplers Improve Quality in Diffusion Models for Limited Steps
This is an incremental improvement for researchers and practitioners using diffusion models under computational constraints.
The paper tackles the problem of limited denoising steps in diffusion models by using two parallel samplers to improve image quality, showing that this approach enhances sample quality without requiring fine-tuning or external models.
We consider the situation where we have a limited number of denoising steps, i.e., of evaluations of a diffusion model. We show that two parallel processors or samplers under such limitation can improve the quality of the sampled image. Particularly, the two samplers make denoising steps at successive times, and their information is appropriately integrated in the latent image. Remarkably, our method is simple both conceptually and to implement: it is plug-&-play, model agnostic, and does not require any additional fine-tuning or external models. We test our method with both automated and human evaluations for different diffusion models. We also show that a naive integration of the information from the two samplers lowers sample quality. Finally, we find that adding more parallel samplers does not necessarily improve sample quality.