Choose Your Simulator Wisely: A Review on Open-source Simulators for Autonomous Driving
It provides a taxonomy and recommendations to help researchers choose appropriate simulators, addressing concerns about algorithm validity in simulation experiments for the autonomous driving community.
This paper tackles the problem of assessing the validity and suitability of open-source simulators for autonomous driving by analyzing their evolution, categorizing them based on task applicability, and identifying critical issues and solutions to enhance credibility.
Simulators play a crucial role in autonomous driving, offering significant time, cost, and labor savings. Over the past few years, the number of simulators for autonomous driving has grown substantially. However, there is a growing concern about the validity of algorithms developed and evaluated in simulators, indicating a need for a thorough analysis of the development status of the simulators. To bridge the gap in research, this paper analyzes the evolution of simulators and explains how the functionalities and utilities have developed. Then, the existing simulators are categorized based on their task applicability, providing researchers with a taxonomy to swiftly assess a simulator's suitability for specific tasks. Recommendations for select simulators are presented, considering factors such as accessibility, maintenance status, and quality. Recognizing potential hazards in simulators that could impact the confidence of simulation experiments, the paper dedicates substantial effort to identifying and justifying critical issues in actively maintained open-source simulators. Moreover, the paper reviews potential solutions to address these issues, serving as a guide for enhancing the credibility of simulators.