Osmosis: RGBD Diffusion Prior for Underwater Image Restoration
This addresses the problem of restoring underwater images for applications like marine research, though it is incremental as it builds on existing diffusion priors.
The paper tackles underwater image restoration by training an unconditional diffusion model on RGBD data of natural outdoor scenes and using a novel guidance method based on the underwater image formation model to remove water effects, outperforming state-of-the-art baselines on challenging scenes.
Underwater image restoration is a challenging task because of water effects that increase dramatically with distance. This is worsened by lack of ground truth data of clean scenes without water. Diffusion priors have emerged as strong image restoration priors. However, they are often trained with a dataset of the desired restored output, which is not available in our case. We also observe that using only color data is insufficient, and therefore augment the prior with a depth channel. We train an unconditional diffusion model prior on the joint space of color and depth, using standard RGBD datasets of natural outdoor scenes in air. Using this prior together with a novel guidance method based on the underwater image formation model, we generate posterior samples of clean images, removing the water effects. Even though our prior did not see any underwater images during training, our method outperforms state-of-the-art baselines for image restoration on very challenging scenes. Our code, models and data are available on the project website.