Christoph Thiem

Nico Weber, Christoph Thiem, Ulrich Konigorski

Scenario-based testing is a promising approach to solve the challenge of proving the safe behavior of vehicles equipped with automated driving systems. Since an infinite number of concrete scenarios can theoretically occur in real-world road traffic, the extraction of scenarios relevant in terms of the safety-related behavior of these systems is a key aspect for their successful verification and validation. Therefore, a method for extracting multimodal urban traffic scenarios from naturalistic road traffic data in an unsupervised manner, minimizing the amount of (potentially biased) prior expert knowledge, is proposed. Rather than an (elaborate) rule-based assignment by extracting concrete scenarios into predefined functional scenarios, the presented method deploys an unsupervised machine learning pipeline. The approach allows exploring the unknown nature of the data and their interpretation as test scenarios that experts could not have anticipated. The method is evaluated for naturalistic road traffic data at urban intersections from the inD and the Silicon Valley Intersections datasets. For this purpose, it is analyzed with which clustering approach (K-Means, hierarchical clustering, and DBSCAN) the scenario extraction method performs best (referring to an elaborate rule-based implementation). Subsequently, using hierarchical clustering the results show both a jump in overall accuracy of around 20% when moving from 4 to 5 clusters and a saturation effect starting at 41 clusters with an overall accuracy of 84%. These observations can be a valuable contribution in the context of the trade-off between the number of functional scenarios (i.e., clustering accuracy) and testing effort. Possible reasons for the observed accuracy variations of different clusters, each with a fixed total number of given clusters, are discussed.

Nico Weber, Christoph Thiem, Ulrich Konigorski

While there was great progress regarding the technology and its implementation for vehicles equipped with automated driving systems (ADS), the problem of how to proof their safety as a necessary precondition prior to market launch remains unsolved. One promising solution are scenario-based test approaches; however, there is no commonly accepted way of how to systematically generate and extract the set of relevant scenarios to be tested to sufficiently capture the real-world traffic dynamics, especially for urban operational design domains. Within the scope of this paper, the overall concept of a novel simulation-based toolchain for the development and testing of ADS-equipped vehicles in urban environments is presented. Based on previous work regarding highway environments, the developed novel enhancements aim at empowering the toolchain to be able to deal with the increased complexity due to the more complex road networks with multi-modal interactions of various traffic participants. Based on derived requirements, a thorough explanation of different modules constituting the toolchain is given, showing first results and identified research gaps, respectively. A closer look is taken on two use cases: First, it is investigated whether the toolchain is capable to serve as synthetic data source within the development phase of ADS-equipped vehicles to enrich a scenario database in terms of extent, complexity and impacts of different what-if-scenarios for future mixed traffic. Second, it is analyzed how to combine the individual advantages of real recorded data and an agent-based simulation within a so-called adaptive replay-to-sim approach to support the testing phase of an ADS-equipped vehicle. The developed toolchain contributes to the overarching goal of a commonly accepted methodology for the validation and safety proof of ADS-equipped vehicles, especially in urban environments.