Introducing SLAMBench, a performance and accuracy benchmarking methodology for SLAM
This work addresses the problem of performance-portable benchmarking for robotics and vision researchers, but it is incremental as it builds on existing SLAM methods and datasets.
The authors tackled the challenge of benchmarking performance, accuracy, and energy consumption in dense RGB-D SLAM systems by introducing SLAMBench, a software framework that provides implementations in multiple programming models and uses synthetic datasets for reliable comparisons, resulting in analyses of execution times on various platforms and energy efficiency on an embedded system.
Real-time dense computer vision and SLAM offer great potential for a new level of scene modelling, tracking and real environmental interaction for many types of robot, but their high computational requirements mean that use on mass market embedded platforms is challenging. Meanwhile, trends in low-cost, low-power processing are towards massive parallelism and heterogeneity, making it difficult for robotics and vision researchers to implement their algorithms in a performance-portable way. In this paper we introduce SLAMBench, a publicly-available software framework which represents a starting point for quantitative, comparable and validatable experimental research to investigate trade-offs in performance, accuracy and energy consumption of a dense RGB-D SLAM system. SLAMBench provides a KinectFusion implementation in C++, OpenMP, OpenCL and CUDA, and harnesses the ICL-NUIM dataset of synthetic RGB-D sequences with trajectory and scene ground truth for reliable accuracy comparison of different implementation and algorithms. We present an analysis and breakdown of the constituent algorithmic elements of KinectFusion, and experimentally investigate their execution time on a variety of multicore and GPUaccelerated platforms. For a popular embedded platform, we also present an analysis of energy efficiency for different configuration alternatives.