Large scale evaluation of local image feature detectors on homography datasets
This work provides a more robust evaluation framework for computer vision researchers, though it is incremental as it builds on existing protocols.
The authors tackled the problem of evaluating local image feature detectors by introducing a new evaluation protocol that improves upon standard repeatability measures, and they conducted a large-scale benchmark using the HPatches dataset to assess detectors on viewpoint and illumination invariance, finding that traditional detectors remain competitive with deep-learning alternatives.
We present a large scale benchmark for the evaluation of local feature detectors. Our key innovation is the introduction of a new evaluation protocol which extends and improves the standard detection repeatability measure. The new protocol is better for assessment on a large number of images and reduces the dependency of the results on unwanted distractors such as the number of detected features and the feature magnification factor. Additionally, our protocol provides a comprehensive assessment of the expected performance of detectors under several practical scenarios. Using images from the recently-introduced HPatches dataset, we evaluate a range of state-of-the-art local feature detectors on two main tasks: viewpoint and illumination invariant detection. Contrary to previous detector evaluations, our study contains an order of magnitude more image sequences, resulting in a quantitative evaluation significantly more robust to over-fitting. We also show that traditional detectors are still very competitive when compared to recent deep-learning alternatives.