Good Keypoints for the Two-View Geometry Estimation Problem
This work addresses a fundamental issue in computer vision for applications like 3D reconstruction, though it is incremental as it builds on existing self-supervised feature detectors.
The paper tackled the problem of identifying keypoint properties that improve two-view geometry estimation, proposing a theoretical model that defines good keypoints as repeatable with small measurement error, and introduced the BoNeSS-ST detector which outperforms prior methods on homography estimation and matches them on epipolar geometry.
Local features are essential to many modern downstream applications. Therefore, it is of interest to determine the properties of local features that contribute to the downstream performance for a better design of feature detectors and descriptors. In our work, we propose a new theoretical model for scoring feature points (keypoints) in the context of the two-view geometry estimation problem. The model determines two properties that a good keypoint for solving the homography estimation problem should have: be repeatable and have a small expected measurement error. This result provides key insights into why maximizing the number of correspondences doesn't always lead to better homography estimation accuracy. We use the developed model to design a method that detects keypoints that benefit the homography estimation and introduce the Bounded NeSS-ST (BoNeSS-ST) keypoint detector. The novelty of BoNeSS-ST comes from strong theoretical foundations, a more accurate keypoint scoring due to subpixel refinement and a cost designed for superior robustness to low saliency keypoints. As a result, BoNeSS-ST outperforms prior self-supervised local feature detectors on the planar homography estimation task and is on par with them on the epipolar geometry estimation task.