An Extensible Benchmarking Infrastructure for Motion Planning Algorithms
This provides a standardized evaluation framework for robotics researchers, but it is incremental as it builds on existing libraries and formats.
The paper tackles the lack of characterization for which sampling-based motion planning algorithms suit specific problem classes by developing a benchmarking infrastructure, resulting in an extensible software framework, data formats, and visualization tool integrated with the Open Motion Planning Library.
Sampling-based planning algorithms are the most common probabilistically complete algorithms and are widely used on many robot platforms. Within this class of algorithms, many variants have been proposed over the last 20 years, yet there is still no characterization of which algorithms are well-suited for which classes of problems. This has motivated us to develop a benchmarking infrastructure for motion planning algorithms. It consists of three main components. First, we have created an extensive benchmarking software framework that is included with the Open Motion Planning Library (OMPL), a C++ library that contains implementations of many sampling-based algorithms. Second, we have defined extensible formats for storing benchmark results. The formats are fairly straightforward so that other planning libraries could easily produce compatible output. Finally, we have created an interactive, versatile visualization tool for compact presentation of collected benchmark data. The tool and underlying database facilitate the analysis of performance across benchmark problems and planners.