Does Redundancy in AI Perception Systems Help to Test for Super-Human Automated Driving Performance?
This work addresses the challenge of proving super-human performance in automated driving systems, highlighting that current redundancy strategies may be insufficient due to correlated errors, which is an incremental but important insight for safety testing in the automotive industry.
The paper investigates whether redundancy in AI perception systems can reliably test for super-human automated driving performance, finding that neural networks performing the same vision task exhibit correlated errors even with independent training or different sensor data, limiting the potential reduction in required testing data.
While automated driving is often advertised with better-than-human driving performance, this work reviews that it is nearly impossible to provide direct statistical evidence on the system level that this is actually the case. The amount of labeled data needed would exceed dimensions of present day technical and economical capabilities. A commonly used strategy therefore is the use of redundancy along with the proof of sufficient subsystems' performances. As it is known, this strategy is efficient especially for the case of subsystems operating independently, i.e. the occurrence of errors is independent in a statistical sense. Here, we give some first considerations and experimental evidence that this strategy is not a free ride as the errors of neural networks fulfilling the same computer vision task, at least for some cases, show correlated occurrences of errors. This remains true, if training data, architecture, and training are kept separate or independence is trained using special loss functions. Using data from different sensors (realized by up to five 2D projections of the 3D MNIST data set) in our experiments is more efficiently reducing correlations, however not to an extent that is realizing the potential of reduction of testing data that can be obtained for redundant and statistically independent subsystems.