Quantifying Assurance in Learning-enabled Systems
This work addresses the need for dependable assurance in safety-critical systems using machine learning, representing an incremental extension of prior methods to system-level applications.
The paper tackles the problem of ensuring dependability in learning-enabled systems (LESs) for safety-critical applications by developing a quantitative notion of assurance through probabilistic measures, and demonstrates its utility in an autonomous aviation system for guiding risk mitigation and dynamic assurance cases.
Dependability assurance of systems embedding machine learning(ML) components---so called learning-enabled systems (LESs)---is a key step for their use in safety-critical applications. In emerging standardization and guidance efforts, there is a growing consensus in the value of using assurance cases for that purpose. This paper develops a quantitative notion of assurance that an LES is dependable, as a core component of its assurance case, also extending our prior work that applied to ML components. Specifically, we characterize LES assurance in the form of assurance measures: a probabilistic quantification of confidence that an LES possesses system-level properties associated with functional capabilities and dependability attributes. We illustrate the utility of assurance measures by application to a real world autonomous aviation system, also describing their role both in i) guiding high-level, runtime risk mitigation decisions and ii) as a core component of the associated dynamic assurance case.