GMM-IKRS: Gaussian Mixture Models for Interpretable Keypoint Refinement and Scoring
This addresses the issue for computer vision researchers and practitioners who need to compare keypoint quality across methods, though it is incremental as it builds on existing keypoint detectors.
The paper tackled the problem of uninterpretable scores for learned keypoints in computer vision, proposing a framework that refines keypoints and provides an interpretable score based on repeatability and localization accuracy, leading to improved repeatability and performance in tasks like homography and pose recovery.
The extraction of keypoints in images is at the basis of many computer vision applications, from localization to 3D reconstruction. Keypoints come with a score permitting to rank them according to their quality. While learned keypoints often exhibit better properties than handcrafted ones, their scores are not easily interpretable, making it virtually impossible to compare the quality of individual keypoints across methods. We propose a framework that can refine, and at the same time characterize with an interpretable score, the keypoints extracted by any method. Our approach leverages a modified robust Gaussian Mixture Model fit designed to both reject non-robust keypoints and refine the remaining ones. Our score comprises two components: one relates to the probability of extracting the same keypoint in an image captured from another viewpoint, the other relates to the localization accuracy of the keypoint. These two interpretable components permit a comparison of individual keypoints extracted across different methods. Through extensive experiments we demonstrate that, when applied to popular keypoint detectors, our framework consistently improves the repeatability of keypoints as well as their performance in homography and two/multiple-view pose recovery tasks.