Volume Feature Rendering for Fast Neural Radiance Field Reconstruction
This addresses the efficiency bottleneck in NeRF rendering for applications like 3D reconstruction and view synthesis, offering a significant speed improvement.
The paper tackles the high computational cost of Neural Radiance Fields (NeRFs) by proposing a volume feature rendering method that reduces neural network evaluations to one per pixel, achieving state-of-the-art rendering quality on synthetic and real-world datasets with training times of several minutes.
Neural radiance fields (NeRFs) are able to synthesize realistic novel views from multi-view images captured from distinct positions and perspectives. In NeRF's rendering pipeline, neural networks are used to represent a scene independently or transform queried learnable feature vector of a point to the expected color or density. With the aid of geometry guides either in occupancy grids or proposal networks, the number of neural network evaluations can be reduced from hundreds to dozens in the standard volume rendering framework. Instead of rendering yielded color after neural network evaluation, we propose to render the queried feature vectors of a ray first and then transform the rendered feature vector to the final pixel color by a neural network. This fundamental change to the standard volume rendering framework requires only one single neural network evaluation to render a pixel, which substantially lowers the high computational complexity of the rendering framework attributed to a large number of neural network evaluations. Consequently, we can use a comparably larger neural network to achieve a better rendering quality while maintaining the same training and rendering time costs. Our model achieves the state-of-the-art rendering quality on both synthetic and real-world datasets while requiring a training time of several minutes.