Single Image Reflection Removal with Physically-Based Training Images
This work addresses the challenge of reflection removal in images, which is important for applications like photography and computer vision, but it is incremental as it builds on existing deep learning approaches with improved training data and network design.
The paper tackles the problem of removing reflections from single images by using physically-based rendering to synthesize training data and introducing a backtrack network to handle complex reflection effects, achieving visually and numerically superior results compared to state-of-the-art methods.
Recently, deep learning-based single image reflection separation methods have been exploited widely. To benefit the learning approach, a large number of training image pairs (i.e., with and without reflections) were synthesized in various ways, yet they are away from a physically-based direction. In this paper, physically based rendering is used for faithfully synthesizing the required training images, and a corresponding network structure and loss term are proposed. We utilize existing RGBD/RGB images to estimate meshes, then physically simulate the light transportation between meshes, glass, and lens with path tracing to synthesize training data, which successfully reproduce the spatially variant anisotropic visual effect of glass reflection. For guiding the separation better, we additionally consider a module, backtrack network (BT-net) for backtracking the reflections, which removes complicated ghosting, attenuation, blurred and defocused effect of glass/lens. This enables obtaining a priori information before having the distortion. The proposed method considering additional a priori information with physically simulated training data is validated with various real reflection images and shows visually pleasant and numerical advantages compared with state-of-the-art techniques.