Exploring 3D-aware Latent Spaces for Efficiently Learning Numerous Scenes
This addresses efficiency challenges in 3D scene reconstruction for applications like virtual reality or robotics, but it is incremental as it builds on existing NeRF and latent space techniques.
The paper tackles the problem of scaling Neural Radiance Fields (NeRFs) to learn many semantically-similar scenes by reducing training time and memory costs, achieving a 44% reduction in per-scene memory and an 86% reduction in per-scene time for 1000 scenes.
We present a method enabling the scaling of NeRFs to learn a large number of semantically-similar scenes. We combine two techniques to improve the required training time and memory cost per scene. First, we learn a 3D-aware latent space in which we train Tri-Plane scene representations, hence reducing the resolution at which scenes are learned. Moreover, we present a way to share common information across scenes, hence allowing for a reduction of model complexity to learn a particular scene. Our method reduces effective per-scene memory costs by 44% and per-scene time costs by 86% when training 1000 scenes. Our project page can be found at https://3da-ae.github.io .