MRPB 1.0: A Unified Benchmark for the Evaluation of Mobile Robot Local Planning Approaches
This provides a standardized tool for researchers and industry users to compare and select local planners, but it is incremental as it builds on existing evaluation needs without introducing new planning methods.
The authors introduced MRPB 1.0, a unified benchmark for evaluating mobile robot local planning approaches, designed with various simulation scenarios and metrics to assess safety, efficiency, and smoothness, and demonstrated its application on two open-source planners.
Local planning is one of the key technologies for mobile robots to achieve full autonomy and has been widely investigated. To evaluate mobile robot local planning approaches in a unified and comprehensive way, a mobile robot local planning benchmark called MRPB 1.0 is newly proposed in this paper. The benchmark facilitates both motion planning researchers who want to compare the performance of a new local planner relative to many other state-of-the-art approaches as well as end users in the mobile robotics industry who want to select a local planner that performs best on some problems of interest. We elaborately design various simulation scenarios to challenge the applicability of local planners, including large-scale, partially unknown, and dynamic complex environments. Furthermore, three types of principled evaluation metrics are carefully designed to quantitatively evaluate the performance of local planners, wherein the safety, efficiency, and smoothness of motions are comprehensively considered. We present the application of the proposed benchmark in two popular open-source local planners to show the practicality of the benchmark. In addition, some insights and guidelines about the design and selection of local planners are also provided. The benchmark website contains all data of the designed simulation scenarios, detailed descriptions of these scenarios, and example code.