From NeRFs to Gaussian Splats, and Back
This addresses the trade-off between generalization and rendering speed for robotics applications, offering an incremental improvement by combining existing methods.
The paper tackled the problem of limited-view generalization in robotics by developing a method to convert between neural radiance fields (NeRFs) and Gaussian splatting (GS), achieving superior PSNR, SSIM, and LPIPS on dissimilar views with real-time rendering and minor computational cost.
For robotics applications where there is a limited number of (typically ego-centric) views, parametric representations such as neural radiance fields (NeRFs) generalize better than non-parametric ones such as Gaussian splatting (GS) to views that are very different from those in the training data; GS however can render much faster than NeRFs. We develop a procedure to convert back and forth between the two. Our approach achieves the best of both NeRFs (superior PSNR, SSIM, and LPIPS on dissimilar views, and a compact representation) and GS (real-time rendering and ability for easily modifying the representation); the computational cost of these conversions is minor compared to training the two from scratch.