DAC: Detector-Agnostic Spatial Covariances for Deep Local Features
This work addresses a specific issue in computer vision for applications requiring precise feature matching, though it is incremental as it builds on existing detectors.
The paper tackled the problem of deep visual local feature detectors lacking spatial uncertainty modeling, which leads to suboptimal downstream results, by proposing two post-hoc covariance estimates that improve feature matching and downstream tasks like perspective-n-point solving and motion-only bundle adjustment.
Current deep visual local feature detectors do not model the spatial uncertainty of detected features, producing suboptimal results in downstream applications. In this work, we propose two post-hoc covariance estimates that can be plugged into any pretrained deep feature detector: a simple, isotropic covariance estimate that uses the predicted score at a given pixel location, and a full covariance estimate via the local structure tensor of the learned score maps. Both methods are easy to implement and can be applied to any deep feature detector. We show that these covariances are directly related to errors in feature matching, leading to improvements in downstream tasks, including solving the perspective-n-point problem and motion-only bundle adjustment. Code is available at https://github.com/javrtg/DAC