NeRF-Supervised Feature Point Detection and Description
This addresses a bottleneck in computer vision applications like SLAM and 3D reconstruction by improving training data realism, though it is incremental as it adapts existing methods rather than introducing new ones.
The paper tackles the problem of limited generalizability in learning-based feature point detection and description by using Neural Radiance Fields (NeRFs) to generate realistic multi-view training data, achieving competitive or superior performance on standard benchmarks for tasks like relative pose estimation while requiring significantly less training data and time.
Feature point detection and description is the backbone for various computer vision applications, such as Structure-from-Motion, visual SLAM, and visual place recognition. While learning-based methods have surpassed traditional handcrafted techniques, their training often relies on simplistic homography-based simulations of multi-view perspectives, limiting model generalisability. This paper presents a novel approach leveraging Neural Radiance Fields (NeRFs) to generate a diverse and realistic dataset consisting of indoor and outdoor scenes. Our proposed methodology adapts state-of-the-art feature detectors and descriptors for training on multi-view NeRF-synthesised data, with supervision achieved through perspective projective geometry. Experiments demonstrate that the proposed methodology achieves competitive or superior performance on standard benchmarks for relative pose estimation, point cloud registration, and homography estimation while requiring significantly less training data and time compared to existing approaches.