RGB cameras failures and their effects in autonomous driving applications
This addresses safety risks for autonomous driving systems by analyzing how camera failures affect image-based applications, though it is incremental as it builds on existing failure analysis and testing methods.
The paper tackles the problem of RGB camera failures in autonomous driving by defining failure modes and analyzing their effects, then tests six object detectors and a simulator agent with generated failed images to quantify misbehaviors and safety risks.
RGB cameras are one of the most relevant sensors for autonomous driving applications. It is undeniable that failures of vehicle cameras may compromise the autonomous driving task, possibly leading to unsafe behaviors when images that are subsequently processed by the driving system are altered. To support the definition of safe and robust vehicle architectures and intelligent systems, in this paper we define the failure modes of a vehicle camera, together with an analysis of effects and known mitigations. Further, we build a software library for the generation of the corresponding failed images and we feed them to six object detectors for mono and stereo cameras and to the self-driving agent of an autonomous driving simulator. The resulting misbehaviors with respect to operating with clean images allow a better understanding of failures effects and the related safety risks in image-based applications.