MAOMaps: A Photo-Realistic Benchmark For vSLAM and Map Merging Quality Assessment
This addresses the need for standardized evaluation in robotics, particularly for researchers and developers working on vSLAM and map merging, though it is incremental as it builds on existing benchmarking efforts.
The authors introduced MAOMaps, a photo-realistic benchmark for evaluating vSLAM and map merging algorithms, providing a dataset with ground truth for both localization and mapping, along with tools for automatic assessment. They developed a novel method for finding correspondences between SLAM-built and ground-truth maps that considers the SLAM context, and the benchmark is open-sourced and ROS-compatible.
Running numerous experiments in simulation is a necessary step before deploying a control system on a real robot. In this paper we introduce a novel benchmark that is aimed at quantitatively evaluating the quality of vision-based simultaneous localization and mapping (vSLAM) and map merging algorithms. The benchmark consists of both a dataset and a set of tools for automatic evaluation. The dataset is photo-realistic and provides both the localization and the map ground truth data. This makes it possible to evaluate not only the localization part of the SLAM pipeline but the mapping part as well. To compare the vSLAM-built maps and the ground-truth ones we introduce a novel way to find correspondences between them that takes the SLAM context into account (as opposed to other approaches like nearest neighbors). The benchmark is ROS-compatable and is open-sourced to the community. The data and the code are available at: \texttt{github.com/CnnDepth/MAOMaps}.