ZeroRF: Fast Sparse View 360° Reconstruction with Zero Pretraining
This addresses the problem of efficient 3D reconstruction from limited views for applications like 3D content generation, though it appears incremental as it builds on existing NeRF methods.
ZeroRF tackles sparse view 360° reconstruction in neural fields by integrating a tailored Deep Image Prior into a factorized NeRF representation, achieving state-of-the-art results on benchmark datasets without pretraining.
We present ZeroRF, a novel per-scene optimization method addressing the challenge of sparse view 360° reconstruction in neural field representations. Current breakthroughs like Neural Radiance Fields (NeRF) have demonstrated high-fidelity image synthesis but struggle with sparse input views. Existing methods, such as Generalizable NeRFs and per-scene optimization approaches, face limitations in data dependency, computational cost, and generalization across diverse scenarios. To overcome these challenges, we propose ZeroRF, whose key idea is to integrate a tailored Deep Image Prior into a factorized NeRF representation. Unlike traditional methods, ZeroRF parametrizes feature grids with a neural network generator, enabling efficient sparse view 360° reconstruction without any pretraining or additional regularization. Extensive experiments showcase ZeroRF's versatility and superiority in terms of both quality and speed, achieving state-of-the-art results on benchmark datasets. ZeroRF's significance extends to applications in 3D content generation and editing. Project page: https://sarahweiii.github.io/zerorf/