Search-based Test-Case Generation by Monitoring Responsibility Safety Rules
This work addresses the need for rigorous testing in automated vehicles, but it is incremental as it builds on existing monitoring and falsification techniques.
The authors tackled the problem of generating qualified test data for training and testing automated vehicle controllers by proposing a method that screens simulation-based driving tests using Responsibility Sensitive Safety rules, resulting in a systematic automated procedure that filters out random tests not meeting safety assumptions.
The safety of Automated Vehicles (AV) as Cyber-Physical Systems (CPS) depends on the safety of their consisting modules (software and hardware) and their rigorous integration. Deep Learning is one of the dominant techniques used for perception, prediction, and decision making in AVs. The accuracy of predictions and decision-making is highly dependant on the tests used for training their underlying deep-learning. In this work, we propose a method for screening and classifying simulation-based driving test data to be used for training and testing controllers. Our method is based on monitoring and falsification techniques, which lead to a systematic automated procedure for generating and selecting qualified test data. We used Responsibility Sensitive Safety (RSS) rules as our qualifier specifications to filter out the random tests that do not satisfy the RSS assumptions. Therefore, the remaining tests cover driving scenarios that the controlled vehicle does not respond safely to its environment. Our framework is distributed with the publicly available S-TALIRO and Sim-ATAV tools.