Assurance Monitoring of Cyber-Physical Systems with Machine Learning Components
This addresses safety hazards for engineers integrating machine learning into cyber-physical systems, but it is incremental as it applies an existing framework to a specific domain.
The paper tackles the problem of ensuring safety in cyber-physical systems with machine learning components by using conformal prediction for assurance monitoring, achieving well-calibrated error rates and a small number of alarms in empirical evaluations on traffic sign recognition and robot navigation datasets.
Machine learning components such as deep neural networks are used extensively in Cyber-Physical Systems (CPS). However, they may introduce new types of hazards that can have disastrous consequences and need to be addressed for engineering trustworthy systems. Although deep neural networks offer advanced capabilities, they must be complemented by engineering methods and practices that allow effective integration in CPS. In this paper, we investigate how to use the conformal prediction framework for assurance monitoring of CPS with machine learning components. In order to handle high-dimensional inputs in real-time, we compute nonconformity scores using embedding representations of the learned models. By leveraging conformal prediction, the approach provides well-calibrated confidence and can allow monitoring that ensures a bounded small error rate while limiting the number of inputs for which an accurate prediction cannot be made. Empirical evaluation results using the German Traffic Sign Recognition Benchmark and a robot navigation dataset demonstrate that the error rates are well-calibrated while the number of alarms is small. The method is computationally efficient, and therefore, the approach is promising for assurance monitoring of CPS.