nerf2nerf: Pairwise Registration of Neural Radiance Fields
This addresses the challenge of aligning neural scene representations for applications in computer vision and graphics, representing an incremental advancement by adapting existing registration methods to a new type of data.
The paper tackles the problem of pairwise registration of Neural Radiance Fields (NeRF) by introducing a technique that extends classical optimization-based local registration to operate on neural 3D scene representations, using a 'surface field' to achieve invariance to illumination and demonstrating effectiveness on synthetic and real datasets.
We introduce a technique for pairwise registration of neural fields that extends classical optimization-based local registration (i.e. ICP) to operate on Neural Radiance Fields (NeRF) -- neural 3D scene representations trained from collections of calibrated images. NeRF does not decompose illumination and color, so to make registration invariant to illumination, we introduce the concept of a ''surface field'' -- a field distilled from a pre-trained NeRF model that measures the likelihood of a point being on the surface of an object. We then cast nerf2nerf registration as a robust optimization that iteratively seeks a rigid transformation that aligns the surface fields of the two scenes. We evaluate the effectiveness of our technique by introducing a dataset of pre-trained NeRF scenes -- our synthetic scenes enable quantitative evaluations and comparisons to classical registration techniques, while our real scenes demonstrate the validity of our technique in real-world scenarios. Additional results available at: https://nerf2nerf.github.io