FrugalNeRF: Fast Convergence for Extreme Few-shot Novel View Synthesis without Learned Priors
This addresses the challenge of efficient and accurate 3D scene reconstruction in few-shot scenarios for computer vision applications, representing an incremental improvement over existing methods.
The paper tackles the problem of overfitting and long training times in Neural Radiance Fields (NeRF) for extreme few-shot novel view synthesis by introducing FrugalNeRF, which uses weight-sharing voxels and a cross-scale geometric adaptation scheme, resulting in outperforming other methods and significantly reducing training time on datasets like LLFF, DTU, and RealEstate-10K.
Neural Radiance Fields (NeRF) face significant challenges in extreme few-shot scenarios, primarily due to overfitting and long training times. Existing methods, such as FreeNeRF and SparseNeRF, use frequency regularization or pre-trained priors but struggle with complex scheduling and bias. We introduce FrugalNeRF, a novel few-shot NeRF framework that leverages weight-sharing voxels across multiple scales to efficiently represent scene details. Our key contribution is a cross-scale geometric adaptation scheme that selects pseudo ground truth depth based on reprojection errors across scales. This guides training without relying on externally learned priors, enabling full utilization of the training data. It can also integrate pre-trained priors, enhancing quality without slowing convergence. Experiments on LLFF, DTU, and RealEstate-10K show that FrugalNeRF outperforms other few-shot NeRF methods while significantly reducing training time, making it a practical solution for efficient and accurate 3D scene reconstruction.