SqueezeNeRF: Further factorized FastNeRF for memory-efficient inference
This work addresses memory inefficiency in real-time novel view generation for applications like VR/AR, but it is incremental as it builds on FastNeRF's factorization approach.
The paper tackled the problem of high memory usage in real-time Neural Radiance Fields (NeRF) inference by proposing SqueezeNeRF, which reduces cache size by over 60 times compared to FastNeRF while maintaining rendering speeds above 190 frames per second.
Neural Radiance Fields (NeRF) has emerged as the state-of-the-art method for novel view generation of complex scenes, but is very slow during inference. Recently, there have been multiple works on speeding up NeRF inference, but the state of the art methods for real-time NeRF inference rely on caching the neural network output, which occupies several giga-bytes of disk space that limits their real-world applicability. As caching the neural network of original NeRF network is not feasible, Garbin et al. proposed "FastNeRF" which factorizes the problem into 2 sub-networks - one which depends only on the 3D coordinate of a sample point and one which depends only on the 2D camera viewing direction. Although this factorization enables them to reduce the cache size and perform inference at over 200 frames per second, the memory overhead is still substantial. In this work, we propose SqueezeNeRF, which is more than 60 times memory-efficient than the sparse cache of FastNeRF and is still able to render at more than 190 frames per second on a high spec GPU during inference.