Levelling the Playing Field: A Comprehensive Comparison of Visual Place Recognition Approaches under Changing Conditions
This work addresses the problem of inconsistent evaluation in Visual Place Recognition research, which is important for robotics and autonomous systems researchers, though it is incremental as it focuses on benchmarking rather than proposing new methods.
This paper comprehensively evaluates 10 state-of-the-art Visual Place Recognition (VPR) methods using three standardized metrics—matching performance, matching time, and memory footprint—to address the difficulty in comparing approaches due to varied datasets and assessment methodologies. The analysis provides an up-to-date snapshot of the strengths and weaknesses of contemporary VPR techniques, aiming to unify the field and enable better future progress.
In recent years there has been significant improvement in the capability of Visual Place Recognition (VPR) methods, building on the success of both hand-crafted and learnt visual features, temporal filtering and usage of semantic scene information. The wide range of approaches and the relatively recent growth in interest in the field has meant that a wide range of datasets and assessment methodologies have been proposed, often with a focus only on precision-recall type metrics, making comparison difficult. In this paper we present a comprehensive approach to evaluating the performance of 10 state-of-the-art recently-developed VPR techniques, which utilizes three standardized metrics: (a) Matching Performance b) Matching Time c) Memory Footprint. Together this analysis provides an up-to-date and widely encompassing snapshot of the various strengths and weaknesses of contemporary approaches to the VPR problem. The aim of this work is to help move this particular research field towards a more mature and unified approach to the problem, enabling better comparison and hence more progress to be made in future research.