Video Waterdrop Removal via Spatio-Temporal Fusion in Driving Scenes
This addresses a safety-critical issue for autonomous driving systems by removing visual obstructions that could lead to accidents, though it is incremental as it builds on existing video restoration methods.
The paper tackles the problem of waterdrops on windshields obstructing vision in driving scenes by proposing an attention-based framework that fuses spatio-temporal representations from multiple frames to restore occluded visual information, achieving the best waterdrop removal performance in complex real-world driving scenes.
The waterdrops on windshields during driving can cause severe visual obstructions, which may lead to car accidents. Meanwhile, the waterdrops can also degrade the performance of a computer vision system in autonomous driving. To address these issues, we propose an attention-based framework that fuses the spatio-temporal representations from multiple frames to restore visual information occluded by waterdrops. Due to the lack of training data for video waterdrop removal, we propose a large-scale synthetic dataset with simulated waterdrops in complex driving scenes on rainy days. To improve the generality of our proposed method, we adopt a cross-modality training strategy that combines synthetic videos and real-world images. Extensive experiments show that our proposed method can generalize well and achieve the best waterdrop removal performance in complex real-world driving scenes.