A Language for Autonomous Vehicles Testing Oracles
This addresses the need for standardized and consistent testing methods for autonomous vehicles, which is crucial for safety and reliability in the automotive industry, though it is incremental as it builds on existing oracle concepts.
The authors tackled the problem of inconsistent and ad-hoc testing oracles for autonomous vehicles by proposing a domain-specific language that allows defining oracles independently of AV solutions and simulators. They demonstrated that existing oracles rank AV solutions significantly differently, highlighting inconsistencies in evaluation.
Testing autonomous vehicles (AVs) requires complex oracles to determine if the AVs behavior conforms with specifications and humans' expectations. Available open source oracles are tightly embedded in the AV simulation software and are developed and implemented in an ad hoc way. We propose a domain specific language that enables defining oracles independent of the AV solutions and the simulator. A testing analyst can encode safety, liveness, timeliness and temporal properties in our language. To show the expressiveness of our language we implement three different types of available oracles. We find that the same AV solutions may be ranked significantly differently across existing oracles, thus existing oracles do not evaluate AVs in a consistent manner.