Learning Run-time Safety Monitors for Machine Learning Components
This addresses safety assurance for autonomous systems, but it is incremental as it builds on existing monitoring concepts with a specific method.
The paper tackles the problem of ensuring safety for machine learning components in autonomous systems by developing run-time safety monitors that predict safety risks when ground truth is unavailable, demonstrating viability through initial experiments on speed sign datasets.
For machine learning components used as part of autonomous systems (AS) in carrying out critical tasks it is crucial that assurance of the models can be maintained in the face of post-deployment changes (such as changes in the operating environment of the system). A critical part of this is to be able to monitor when the performance of the model at runtime (as a result of changes) poses a safety risk to the system. This is a particularly difficult challenge when ground truth is unavailable at runtime. In this paper we introduce a process for creating safety monitors for ML components through the use of degraded datasets and machine learning. The safety monitor that is created is deployed to the AS in parallel to the ML component to provide a prediction of the safety risk associated with the model output. We demonstrate the viability of our approach through some initial experiments using publicly available speed sign datasets.