Ensembling Uncertainty Measures to Improve Safety of Black-Box Classifiers
This addresses safety issues in ML systems by preventing critical failures from misclassifications, though it is incremental as it builds on existing uncertainty measures.
The paper tackles the problem of misclassifications in black-box classifiers by proposing SPROUT, a safety wrapper that uses ensembles of uncertainty measures to detect and block misclassifications, transforming them into manageable data omission failures, with experiments showing it identifies a huge fraction of misclassifications and detects all in specific cases.
Machine Learning (ML) algorithms that perform classification may predict the wrong class, experiencing misclassifications. It is well-known that misclassifications may have cascading effects on the encompassing system, possibly resulting in critical failures. This paper proposes SPROUT, a Safety wraPper thROugh ensembles of UncertainTy measures, which suspects misclassifications by computing uncertainty measures on the inputs and outputs of a black-box classifier. If a misclassification is detected, SPROUT blocks the propagation of the output of the classifier to the encompassing system. The resulting impact on safety is that SPROUT transforms erratic outputs (misclassifications) into data omission failures, which can be easily managed at the system level. SPROUT has a broad range of applications as it fits binary and multi-class classification, comprising image and tabular datasets. We experimentally show that SPROUT always identifies a huge fraction of the misclassifications of supervised classifiers, and it is able to detect all misclassifications in specific cases. SPROUT implementation contains pre-trained wrappers, it is publicly available and ready to be deployed with minimal effort.