Harnessing Joint Rain-/Detail-aware Representations to Eliminate Intricate Rains
This work addresses the challenge of effectively removing rain from images for computer vision applications, but it is incremental as it builds on existing deraining methods by introducing new mechanisms for better handling diverse datasets.
The paper tackles the problem of image deraining by addressing suboptimal results from training on mixed datasets, proposing a method that uses joint rain-/detail-aware representations and a Context-based Instance-level Modulation mechanism to improve deraining performance for CNN and Transformer models, with experiments showing enhanced ability, especially with real-world data.
Recent advances in image deraining have focused on training powerful models on mixed multiple datasets comprising diverse rain types and backgrounds. However, this approach tends to overlook the inherent differences among rainy images, leading to suboptimal results. To overcome this limitation, we focus on addressing various rainy images by delving into meaningful representations that encapsulate both the rain and background components. Leveraging these representations as instructive guidance, we put forth a Context-based Instance-level Modulation (CoI-M) mechanism adept at efficiently modulating CNN- or Transformer-based models. Furthermore, we devise a rain-/detail-aware contrastive learning strategy to help extract joint rain-/detail-aware representations. By integrating CoI-M with the rain-/detail-aware Contrastive learning, we develop CoIC, an innovative and potent algorithm tailored for training models on mixed datasets. Moreover, CoIC offers insight into modeling relationships of datasets, quantitatively assessing the impact of rain and details on restoration, and unveiling distinct behaviors of models given diverse inputs. Extensive experiments validate the efficacy of CoIC in boosting the deraining ability of CNN and Transformer models. CoIC also enhances the deraining prowess remarkably when real-world dataset is included.