Progressively-connected Light Field Network for Efficient View Synthesis
This addresses the problem of efficient and high-quality view synthesis for computer vision applications, with incremental improvements over existing neural light field and NeRF-like approaches.
The paper tackles novel view synthesis for complex forward-facing scenes by proposing a Progressively-connected Light Field network (ProLiF), which encodes a 4D light field and uses progressive training with regularization to improve multi-view consistency. It achieves significantly better rendering quality than vanilla neural light fields and comparable results to NeRF-like methods on the LLFF and Shiny Object datasets.
This paper presents a Progressively-connected Light Field network (ProLiF), for the novel view synthesis of complex forward-facing scenes. ProLiF encodes a 4D light field, which allows rendering a large batch of rays in one training step for image- or patch-level losses. Directly learning a neural light field from images has difficulty in rendering multi-view consistent images due to its unawareness of the underlying 3D geometry. To address this problem, we propose a progressive training scheme and regularization losses to infer the underlying geometry during training, both of which enforce the multi-view consistency and thus greatly improves the rendering quality. Experiments demonstrate that our method is able to achieve significantly better rendering quality than the vanilla neural light fields and comparable results to NeRF-like rendering methods on the challenging LLFF dataset and Shiny Object dataset. Moreover, we demonstrate better compatibility with LPIPS loss to achieve robustness to varying light conditions and CLIP loss to control the rendering style of the scene. Project page: https://totoro97.github.io/projects/prolif.