StructNeRF: Neural Radiance Fields for Indoor Scenes with Structural Hints
This addresses the challenge of sparse-view 3D reconstruction for indoor scenes, offering an incremental improvement over existing NeRF methods.
The paper tackles the problem of poor novel view synthesis quality in Neural Radiance Fields (NeRF) with sparse input views for indoor scenes, and proposes StructNeRF, which leverages structural hints to improve geometry and achieves state-of-the-art results on real-world datasets.
Neural Radiance Fields (NeRF) achieve photo-realistic view synthesis with densely captured input images. However, the geometry of NeRF is extremely under-constrained given sparse views, resulting in significant degradation of novel view synthesis quality. Inspired by self-supervised depth estimation methods, we propose StructNeRF, a solution to novel view synthesis for indoor scenes with sparse inputs. StructNeRF leverages the structural hints naturally embedded in multi-view inputs to handle the unconstrained geometry issue in NeRF. Specifically, it tackles the texture and non-texture regions respectively: a patch-based multi-view consistent photometric loss is proposed to constrain the geometry of textured regions; for non-textured ones, we explicitly restrict them to be 3D consistent planes. Through the dense self-supervised depth constraints, our method improves both the geometry and the view synthesis performance of NeRF without any additional training on external data. Extensive experiments on several real-world datasets demonstrate that StructNeRF surpasses state-of-the-art methods for indoor scenes with sparse inputs both quantitatively and qualitatively.