On Robustness of Lane Detection Models to Physical-World Adversarial Attacks in Autonomous Driving
This highlights a critical gap in evaluation methods for lane detection, which is essential for improving safety in autonomous driving systems.
The study evaluated the robustness of state-of-the-art lane detection models to physical-world adversarial attacks in autonomous driving, finding that conventional metrics like accuracy and F1 score fail to reflect real-world performance, with the most robust model on these metrics being the least robust in end-to-end scenarios.
After the 2017 TuSimple Lane Detection Challenge, its evaluation based on accuracy and F1 score has become the de facto standard to measure the performance of lane detection methods. In this work, we conduct the first large-scale empirical study to evaluate the robustness of state-of-the-art lane detection methods under physical-world adversarial attacks in autonomous driving. We evaluate 4 major types of lane detection approaches with the conventional evaluation and end-to-end evaluation in autonomous driving scenarios and then discuss the security proprieties of each lane detection model. We demonstrate that the conventional evaluation fails to reflect the robustness in end-to-end autonomous driving scenarios. Our results show that the most robust model on the conventional metrics is the least robust in the end-to-end evaluation. Although the competition dataset and its metrics have played a substantial role in developing performant lane detection methods along with the rapid development of deep neural networks, the conventional evaluation is becoming obsolete and the gap between the metrics and practicality is critical. We hope that our study will help the community make further progress in building a more comprehensive framework to evaluate lane detection models.