Heavy Rain Image Restoration: Integrating Physics Model and Conditional Adversarial Learning
This work addresses a specific challenge in computer vision for applications like autonomous driving or surveillance, but it is incremental as it builds on existing deraining methods.
The paper tackles the problem of restoring heavy rain images, where existing methods fail due to dense rain streaks and veiling effects, and proposes a two-stage network that integrates a physics-based model with a depth-guided GAN, achieving state-of-the-art performance on real rain image data with visually clean results.
Most deraining works focus on rain streaks removal but they cannot deal adequately with heavy rain images. In heavy rain, streaks are strongly visible, dense rain accumulation or rain veiling effect significantly washes out the image, further scenes are relatively more blurry, etc. In this paper, we propose a novel method to address these problems. We put forth a 2-stage network: a physics-based backbone followed by a depth-guided GAN refinement. The first stage estimates the rain streaks, the transmission, and the atmospheric light governed by the underlying physics. To tease out these components more reliably, a guided filtering framework is used to decompose the image into its low- and high-frequency components. This filtering is guided by a rain-free residue image --- its content is used to set the passbands for the two channels in a spatially-variant manner so that the background details do not get mixed up with the rain-streaks. For the second stage, the refinement stage, we put forth a depth-guided GAN to recover the background details failed to be retrieved by the first stage, as well as correcting artefacts introduced by that stage. We have evaluated our method against the state of the art methods. Extensive experiments show that our method outperforms them on real rain image data, recovering visually clean images with good details.