SG-NeRF: Neural Surface Reconstruction with Scene Graph Optimization
This addresses a critical bottleneck in real-world 3D modeling applications where accurate camera poses are often unavailable, offering a robust solution for handling outliers.
The paper tackles the problem of 3D surface reconstruction from images using Neural Radiance Fields (NeRFs) when camera poses are noisy or contain outliers, proposing a method that optimizes radiance fields with scene graphs to mitigate outlier influence and demonstrating effectiveness and superiority over existing approaches in robustness and quality.
3D surface reconstruction from images is essential for numerous applications. Recently, Neural Radiance Fields (NeRFs) have emerged as a promising framework for 3D modeling. However, NeRFs require accurate camera poses as input, and existing methods struggle to handle significantly noisy pose estimates (i.e., outliers), which are commonly encountered in real-world scenarios. To tackle this challenge, we present a novel approach that optimizes radiance fields with scene graphs to mitigate the influence of outlier poses. Our method incorporates an adaptive inlier-outlier confidence estimation scheme based on scene graphs, emphasizing images of high compatibility with the neighborhood and consistency in the rendering quality. We also introduce an effective intersection-over-union (IoU) loss to optimize the camera pose and surface geometry, together with a coarse-to-fine strategy to facilitate the training. Furthermore, we propose a new dataset containing typical outlier poses for a detailed evaluation. Experimental results on various datasets consistently demonstrate the effectiveness and superiority of our method over existing approaches, showcasing its robustness in handling outliers and producing high-quality 3D reconstructions. Our code and data are available at: \url{https://github.com/Iris-cyy/SG-NeRF}.