Light Field View Synthesis via Aperture Disparity and Warping Confidence Map
This work addresses view synthesis for computer vision applications, but it is incremental as it builds on existing learning-based approaches with specific improvements for occlusion handling.
The paper tackles the problem of synthesizing novel views from sparse images by addressing inconsistencies due to occlusions, using a method that combines aperture disparity maps and warping confidence maps in a CNN, achieving better performance than state-of-the-art techniques on diverse light field scenes.
This paper presents a learning-based approach to synthesize the view from an arbitrary camera position given a sparse set of images. A key challenge for this novel view synthesis arises from the reconstruction process, when the views from different input images may not be consistent due to obstruction in the light path. We overcome this by jointly modeling the epipolar property and occlusion in designing a convolutional neural network. We start by defining and computing the aperture disparity map, which approximates the parallax and measures the pixel-wise shift between two views. While this relates to free-space rendering and can fail near the object boundaries, we further develop a warping confidence map to address pixel occlusion in these challenging regions. The proposed method is evaluated on diverse real-world and synthetic light field scenes, and it shows better performance over several state-of-the-art techniques.