Fast Training of Neural Lumigraph Representations using Meta Learning
This addresses the efficiency bottleneck in neural rendering for computer vision applications, offering a practical improvement for tasks like virtual reality or 3D reconstruction.
The paper tackles the slow training and rendering of neural scene representations for novel view synthesis by introducing MetaNLR++, which uses meta learning to learn priors that reduce training time to minutes while achieving similar or better synthesis quality than competing methods.
Novel view synthesis is a long-standing problem in machine learning and computer vision. Significant progress has recently been made in developing neural scene representations and rendering techniques that synthesize photorealistic images from arbitrary views. These representations, however, are extremely slow to train and often also slow to render. Inspired by neural variants of image-based rendering, we develop a new neural rendering approach with the goal of quickly learning a high-quality representation which can also be rendered in real-time. Our approach, MetaNLR++, accomplishes this by using a unique combination of a neural shape representation and 2D CNN-based image feature extraction, aggregation, and re-projection. To push representation convergence times down to minutes, we leverage meta learning to learn neural shape and image feature priors which accelerate training. The optimized shape and image features can then be extracted using traditional graphics techniques and rendered in real time. We show that MetaNLR++ achieves similar or better novel view synthesis results in a fraction of the time that competing methods require.