Is the Rush to Machine Learning Jeopardizing Safety? Results of a Survey
This highlights a critical gap in ensuring safety for systems like autonomous vehicles and medical devices, though it is incremental as it surveys existing literature rather than proposing new solutions.
The paper tackles the problem of machine learning deployment in safety-critical systems potentially compromising safety due to inadequate research on safety evaluation, finding through a survey that research effort on applying ML significantly outweighs that on safety assessment.
Machine learning (ML) is finding its way into safety-critical systems (SCS). Current safety standards and practice were not designed to cope with ML techniques, and it is difficult to be confident that SCSs that contain ML components are safe. Our hypothesis was that there has been a rush to deploy ML techniques at the expense of a thorough examination as to whether the use of ML techniques introduces safety problems that we are not yet adequately able to detect and mitigate against. We thus conducted a targeted literature survey to determine the research effort that has been expended in applying ML to SCS compared with that spent on evaluating the safety of SCSs that deploy ML components. This paper presents the (surprising) results of the survey.