Benchmarking Image Perturbations for Testing Automated Driving Assistance Systems
It addresses safety-critical failures in ADAS perception systems for autonomous vehicles, but is incremental as it builds on existing perturbation techniques.
This study evaluated 38 categories of image perturbations to test the robustness of Advanced Driver Assistance Systems (ADAS) based on deep neural networks, finding that all categories exposed issues and that data augmentation and continuous learning significantly improved performance in unseen environments.
Advanced Driver Assistance Systems (ADAS) based on deep neural networks (DNNs) are widely used in autonomous vehicles for critical perception tasks such as object detection, semantic segmentation, and lane recognition. However, these systems are highly sensitive to input variations, such as noise and changes in lighting, which can compromise their effectiveness and potentially lead to safety-critical failures. This study offers a comprehensive empirical evaluation of image perturbations, techniques commonly used to assess the robustness of DNNs, to validate and improve the robustness and generalization of ADAS perception systems. We first conducted a systematic review of the literature, identifying 38 categories of perturbations. Next, we evaluated their effectiveness in revealing failures in two different ADAS, both at the component and at the system level. Finally, we explored the use of perturbation-based data augmentation and continuous learning strategies to improve ADAS adaptation to new operational design domains. Our results demonstrate that all categories of image perturbations successfully expose robustness issues in ADAS and that the use of dataset augmentation and continuous learning significantly improves ADAS performance in novel, unseen environments.