Seeing through Unclear Glass: Occlusion Removal with One Shot
This addresses the challenge of image restoration for real-world occlusions in photography, though it is incremental as it builds on existing deep learning approaches with a novel adaptation mechanism.
The paper tackles the problem of restoring images degraded by various contaminants on window glass, such as muddy water and dirt, by proposing a one-shot test-time adaptation model that outperforms state-of-the-art methods on realistic and unseen contaminated images.
Images taken through window glass are often degraded by contaminants adhered to the glass surfaces. Such contaminants cause occlusions that attenuate the incoming light and scatter stray light towards the camera. Most of existing deep learning methods for neutralizing the effects of contaminated glasses relied on synthetic training data. Few researchers used real degraded and clean image pairs, but they only considered removing or alleviating the effects of rain drops on glasses. This paper is concerned with the more challenging task of learning the restoration of images taken through glasses contaminated by a wide range of occluders, including muddy water, dirt and other small foreign particles found in reality. To facilitate the learning task we have gone to a great length to acquire real paired images with and without glass contaminants. More importantly, we propose an all-in-one model to neutralize contaminants of different types by utilizing the one-shot test-time adaptation mechanism. It involves a self-supervised auxiliary learning task to update the trained model for the unique occlusion type of each test image. Experimental results show that the proposed method outperforms the state-of-the-art methods quantitatively and qualitatively in cleaning realistic contaminated images, especially the unseen ones.