Bi-Noising Diffusion: Towards Conditional Diffusion Models with Generative Restoration Priors
This work addresses the issue of generating unrealistic images in conditional diffusion models for computer vision applications, representing an incremental improvement by incorporating generative restoration priors.
The paper tackles the problem of unrealistic outputs in conditional diffusion models, such as color shifts and textures, by introducing a method that uses a pretrained unconditional diffusion model as a regularizer to reduce distribution divergence during sampling. The approach shows effectiveness in tasks like super-resolution, colorization, turbulence removal, and image-deraining, with improvements suggesting it can serve as a general plugin for enhancing conditional models.
Conditional diffusion probabilistic models can model the distribution of natural images and can generate diverse and realistic samples based on given conditions. However, oftentimes their results can be unrealistic with observable color shifts and textures. We believe that this issue results from the divergence between the probabilistic distribution learned by the model and the distribution of natural images. The delicate conditions gradually enlarge the divergence during each sampling timestep. To address this issue, we introduce a new method that brings the predicted samples to the training data manifold using a pretrained unconditional diffusion model. The unconditional model acts as a regularizer and reduces the divergence introduced by the conditional model at each sampling step. We perform comprehensive experiments to demonstrate the effectiveness of our approach on super-resolution, colorization, turbulence removal, and image-deraining tasks. The improvements obtained by our method suggest that the priors can be incorporated as a general plugin for improving conditional diffusion models.