VF-NeRF: Viewshed Fields for Rigid NeRF Registration
This addresses a fundamental computer vision problem for 3D scene alignment in NeRF-based applications, representing an incremental advancement by applying a novel method to a known bottleneck in NeRF registration.
The paper tackles the problem of rigid registration between two Neural Radiance Fields (NeRFs) without given camera positions by introducing Viewshed Fields (VF), an implicit function that estimates the likelihood of 3D points being viewed by original cameras, achieving state-of-the-art results on datasets like LLFF and Objaverese.
3D scene registration is a fundamental problem in computer vision that seeks the best 6-DoF alignment between two scenes. This problem was extensively investigated in the case of point clouds and meshes, but there has been relatively limited work regarding Neural Radiance Fields (NeRF). In this paper, we consider the problem of rigid registration between two NeRFs when the position of the original cameras is not given. Our key novelty is the introduction of Viewshed Fields (VF), an implicit function that determines, for each 3D point, how likely it is to be viewed by the original cameras. We demonstrate how VF can help in the various stages of NeRF registration, with an extensive evaluation showing that VF-NeRF achieves SOTA results on various datasets with different capturing approaches such as LLFF and Objaverese.