PlaNeRF: SVD Unsupervised 3D Plane Regularization for NeRF Large-Scale Scene Reconstruction
This work addresses geometry accuracy issues in NeRF for applications like extrapolated novel view synthesis and HD mapping, representing an incremental improvement over existing regularization methods.
The paper tackles the problem of poor geometry reconstruction in Neural Radiance Fields (NeRF) for large-scale outdoor scenes, particularly in low-texture areas, by proposing an unsupervised plane regularization method using SVD and SSIM, resulting in state-of-the-art rendering quality on the KITTI-360 benchmark.
Neural Radiance Fields (NeRF) enable 3D scene reconstruction from 2D images and camera poses for Novel View Synthesis (NVS). Although NeRF can produce photorealistic results, it often suffers from overfitting to training views, leading to poor geometry reconstruction, especially in low-texture areas. This limitation restricts many important applications which require accurate geometry, such as extrapolated NVS, HD mapping and scene editing. To address this limitation, we propose a new method to improve NeRF's 3D structure using only RGB images and semantic maps. Our approach introduces a novel plane regularization based on Singular Value Decomposition (SVD), that does not rely on any geometric prior. In addition, we leverage the Structural Similarity Index Measure (SSIM) in our loss design to properly initialize the volumetric representation of NeRF. Quantitative and qualitative results show that our method outperforms popular regularization approaches in accurate geometry reconstruction for large-scale outdoor scenes and achieves SoTA rendering quality on the KITTI-360 NVS benchmark.