HPatches: A benchmark and evaluation of handcrafted and learned local descriptors
This provides a more reliable benchmark for researchers in computer vision to compare local descriptors, addressing inconsistencies in the field, though it is incremental as it builds on existing evaluation methods.
The authors tackled the problem of ambiguous and saturated datasets for evaluating local image descriptors by introducing HPatches, a new large dataset with strict evaluation protocols for tasks like matching, retrieval, and classification, showing that normalizing hand-crafted descriptors can boost their performance to match deep learning-based ones in realistic benchmarks.
In this paper, we propose a novel benchmark for evaluating local image descriptors. We demonstrate that the existing datasets and evaluation protocols do not specify unambiguously all aspects of evaluation, leading to ambiguities and inconsistencies in results reported in the literature. Furthermore, these datasets are nearly saturated due to the recent improvements in local descriptors obtained by learning them from large annotated datasets. Therefore, we introduce a new large dataset suitable for training and testing modern descriptors, together with strictly defined evaluation protocols in several tasks such as matching, retrieval and classification. This allows for more realistic, and thus more reliable comparisons in different application scenarios. We evaluate the performance of several state-of-the-art descriptors and analyse their properties. We show that a simple normalisation of traditional hand-crafted descriptors can boost their performance to the level of deep learning based descriptors within a realistic benchmarks evaluation.