Learning Occlusion-Aware View Synthesis for Light Fields
This work addresses the challenge of view synthesis in light fields, particularly for handling occlusions, which is important for applications in computer vision and graphics, but it appears incremental as it builds on existing learning-based approaches.
The authors tackled the problem of synthesizing new views from a light field image using only four corner views, achieving superior performance in handling occlusions and object boundaries compared to state-of-the-art methods, as demonstrated on Lytro light fields and camera array data.
In this work, we present a novel learning-based approach to synthesize new views of a light field image. In particular, given the four corner views of a light field, the presented method estimates any in-between view. We use three sequential convolutional neural networks for feature extraction, scene geometry estimation and view selection. Compared to state-of-the-art approaches, in order to handle occlusions we propose to estimate a different disparity map per view. Jointly with the view selection network, this strategy shows to be the most important to have proper reconstructions near object boundaries. Ablation studies and comparison against the state of the art on Lytro light fields show the superior performance of the proposed method. Furthermore, the method is adapted and tested on light fields with wide baselines acquired with a camera array and, in spite of having to deal with large occluded areas, the proposed approach yields very promising results.