Benchmarking the Physical-world Adversarial Robustness of Vehicle Detection
This work addresses the challenge of benchmarking adversarial robustness for vehicle detection in autonomous driving, but it is incremental as it builds on existing simulation tools and attack methods.
The paper tackled the lack of a unified benchmark for evaluating the physical-world adversarial robustness of vehicle detection models by proposing a data generation pipeline using the CARLA simulator, resulting in findings such as Yolo v6 showing the strongest resistance with only a 6.59% average AP drop and ASA being the most effective attack with a 14.51% average AP reduction.
Adversarial attacks in the physical world can harm the robustness of detection models. Evaluating the robustness of detection models in the physical world can be challenging due to the time-consuming and labor-intensive nature of many experiments. Thus, virtual simulation experiments can provide a solution to this challenge. However, there is no unified detection benchmark based on virtual simulation environment. To address this challenge, we proposed an instant-level data generation pipeline based on the CARLA simulator. Using this pipeline, we generated the DCI dataset and conducted extensive experiments on three detection models and three physical adversarial attacks. The dataset covers 7 continuous and 1 discrete scenes, with over 40 angles, 20 distances, and 20,000 positions. The results indicate that Yolo v6 had strongest resistance, with only a 6.59% average AP drop, and ASA was the most effective attack algorithm with a 14.51% average AP reduction, twice that of other algorithms. Static scenes had higher recognition AP, and results under different weather conditions were similar. Adversarial attack algorithm improvement may be approaching its 'limitation'.