Learning to See Through Obstructions with Layered Decomposition
This addresses a practical problem for computer vision applications in photography and surveillance by providing a robust method for obstruction removal, though it builds incrementally on prior motion-based layer separation techniques.
The paper tackles the problem of removing obstructions like window reflections and fence occlusions from image sequences by developing a learning-based approach that leverages motion differences to recover background and obstruction layers. It demonstrates effectiveness on challenging real-world scenarios, with results showing performance improvements over existing methods.
We present a learning-based approach for removing unwanted obstructions, such as window reflections, fence occlusions, or adherent raindrops, from a short sequence of images captured by a moving camera. Our method leverages motion differences between the background and obstructing elements to recover both layers. Specifically, we alternate between estimating dense optical flow fields of the two layers and reconstructing each layer from the flow-warped images via a deep convolutional neural network. This learning-based layer reconstruction module facilitates accommodating potential errors in the flow estimation and brittle assumptions, such as brightness consistency. We show that the proposed approach learned from synthetically generated data performs well to real images. Experimental results on numerous challenging scenarios of reflection and fence removal demonstrate the effectiveness of the proposed method.