Joint Conditional Diffusion Model for Image Restoration with Mixed Degradations
This addresses the problem of restoring images corrupted by multiple simultaneous weather-related degradations for applications in computer vision, though it appears incremental by building on diffusion models with added constraints.
The paper tackles image restoration under mixed adverse weather degradations by proposing a joint conditional diffusion model guided by physical constraints, achieving superior results over state-of-the-art methods on multi-weather and weather-specific datasets.
Image restoration is rather challenging in adverse weather conditions, especially when multiple degradations occur simultaneously. Blind image decomposition was proposed to tackle this issue, however, its effectiveness heavily relies on the accurate estimation of each component. Although diffusion-based models exhibit strong generative abilities in image restoration tasks, they may generate irrelevant contents when the degraded images are severely corrupted. To address these issues, we leverage physical constraints to guide the whole restoration process, where a mixed degradation model based on atmosphere scattering model is constructed. Then we formulate our Joint Conditional Diffusion Model (JCDM) by incorporating the degraded image and degradation mask to provide precise guidance. To achieve better color and detail recovery results, we further integrate a refinement network to reconstruct the restored image, where Uncertainty Estimation Block (UEB) is employed to enhance the features. Extensive experiments performed on both multi-weather and weather-specific datasets demonstrate the superiority of our method over state-of-the-art competing methods.