SLAMBench2: Multi-Objective Head-to-Head Benchmarking for Visual SLAM
This provides a standardized tool for researchers and developers in robotics and AR to evaluate SLAM systems based on specific requirements like accuracy or energy efficiency, though it is incremental as it builds on existing benchmarking efforts.
The authors tackled the lack of unified benchmarking for visual SLAM algorithms by introducing SLAMBench2, a framework that supports multiple datasets and performance metrics, enabling head-to-head comparisons of systems like ElasticFusion and ORB-SLAM2.
SLAM is becoming a key component of robotics and augmented reality (AR) systems. While a large number of SLAM algorithms have been presented, there has been little effort to unify the interface of such algorithms, or to perform a holistic comparison of their capabilities. This is a problem since different SLAM applications can have different functional and non-functional requirements. For example, a mobile phonebased AR application has a tight energy budget, while a UAV navigation system usually requires high accuracy. SLAMBench2 is a benchmarking framework to evaluate existing and future SLAM systems, both open and close source, over an extensible list of datasets, while using a comparable and clearly specified list of performance metrics. A wide variety of existing SLAM algorithms and datasets is supported, e.g. ElasticFusion, InfiniTAM, ORB-SLAM2, OKVIS, and integrating new ones is straightforward and clearly specified by the framework. SLAMBench2 is a publicly-available software framework which represents a starting point for quantitative, comparable and validatable experimental research to investigate trade-offs across SLAM systems.