Let's See Clearly: Contaminant Artifact Removal for Moving Cameras
This work addresses video quality degradation for users of moving cameras in environments with contaminants, presenting an incremental improvement with a novel framework and dataset.
The paper tackles the problem of removing contaminants like dust and moisture from videos captured by moving cameras, proposing a method that detects attention maps, hallucinates background flow, and uses recurrent restoration to fetch clean pixels from adjacent frames, achieving superior qualitative and quantitative results compared to competitive approaches.
Contaminants such as dust, dirt and moisture adhering to the camera lens can greatly affect the quality and clarity of the resulting image or video. In this paper, we propose a video restoration method to automatically remove these contaminants and produce a clean video. Our approach first seeks to detect attention maps that indicate the regions that need to be restored. In order to leverage the corresponding clean pixels from adjacent frames, we propose a flow completion module to hallucinate the flow of the background scene to the attention regions degraded by the contaminants. Guided by the attention maps and completed flows, we propose a recurrent technique to restore the input frame by fetching clean pixels from adjacent frames. Finally, a multi-frame processing stage is used to further process the entire video sequence in order to enforce temporal consistency. The entire network is trained on a synthetic dataset that approximates the physical lighting properties of contaminant artifacts. This new dataset and our novel framework lead to our method that is able to address different contaminants and outperforms competitive restoration approaches both qualitatively and quantitatively.