DeRF: Decomposed Radiance Fields
This work provides a significant improvement in inference efficiency for NeRF, which is beneficial for researchers and practitioners working with 3D scene rendering and novel view synthesis.
This paper addresses the computational intensity of Neural Radiance Fields (NeRF) by proposing a spatial decomposition technique. By dedicating smaller networks to decomposed parts of a scene, the method achieves up to 3x more efficient inference than NeRF with the same rendering quality, or an improvement of up to 1.0 dB in PSNR for the same inference cost.
With the advent of Neural Radiance Fields (NeRF), neural networks can now render novel views of a 3D scene with quality that fools the human eye. Yet, generating these images is very computationally intensive, limiting their applicability in practical scenarios. In this paper, we propose a technique based on spatial decomposition capable of mitigating this issue. Our key observation is that there are diminishing returns in employing larger (deeper and/or wider) networks. Hence, we propose to spatially decompose a scene and dedicate smaller networks for each decomposed part. When working together, these networks can render the whole scene. This allows us near-constant inference time regardless of the number of decomposed parts. Moreover, we show that a Voronoi spatial decomposition is preferable for this purpose, as it is provably compatible with the Painter's Algorithm for efficient and GPU-friendly rendering. Our experiments show that for real-world scenes, our method provides up to 3x more efficient inference than NeRF (with the same rendering quality), or an improvement of up to 1.0~dB in PSNR (for the same inference cost).