Optimizing SLAM Evaluation Footprint Through Dynamic Range Coverage Analysis of Datasets
This work addresses efficiency in SLAM evaluation for researchers and practitioners, but it is incremental as it optimizes existing processes rather than introducing new SLAM methods.
The paper tackles the problem of redundant evaluation in Simultaneous Localization and Mapping (SLAM) by analyzing dataset dynamic range coverage and proposing a dynamic programming algorithm to select optimal subsets, achieving a reduction in evaluation effort while maintaining coverage.
Simultaneous Localization and Mapping (SLAM) is considered an ever-evolving problem due to its usage in many applications. Evaluation of SLAM is done typically using publicly available datasets which are increasing in number and the level of difficulty. Each dataset provides a certain level of dynamic range coverage that is a key aspect of measuring the robustness and resilience of SLAM. In this paper, we provide a systematic analysis of the dynamic range coverage of datasets based on a number of characterization metrics, and our analysis shows a huge level of redundancy within and between datasets. Subsequently, we propose a dynamic programming (DP) algorithm for eliminating the redundancy in the evaluation process of SLAM by selecting a subset of sequences that matches a single or multiple dynamic range coverage objectives. It is shown that, with the help of dataset characterization and DP selection algorithm, a reduction in the evaluation effort can be achieved while maintaining the same level of coverage. We also study how the evaluation process of a real-world SLAM system can be optimized utilizing the method proposed.