Bayesian Modelling in Practice: Using Uncertainty to Improve Trustworthiness in Medical Applications
This addresses the problem of improving trustworthiness in high-risk medical applications like the ICU for doctors, but it is incremental as it applies an existing Bayesian method to a specific domain.
The paper tackles the problem of machine learning models in the ICU providing only point-estimates without uncertainty, which can lead to catastrophic treatment decisions, and shows that Bayesian modeling with predictive uncertainty can mitigate risk and detect out-of-domain examples, with empirical results on the MIMIC-III dataset demonstrating it prevents errors and identifies out-of-domain patients.
The Intensive Care Unit (ICU) is a hospital department where machine learning has the potential to provide valuable assistance in clinical decision making. Classical machine learning models usually only provide point-estimates and no uncertainty of predictions. In practice, uncertain predictions should be presented to doctors with extra care in order to prevent potentially catastrophic treatment decisions. In this work we show how Bayesian modelling and the predictive uncertainty that it provides can be used to mitigate risk of misguided prediction and to detect out-of-domain examples in a medical setting. We derive analytically a bound on the prediction loss with respect to predictive uncertainty. The bound shows that uncertainty can mitigate loss. Furthermore, we apply a Bayesian Neural Network to the MIMIC-III dataset, predicting risk of mortality of ICU patients. Our empirical results show that uncertainty can indeed prevent potential errors and reliably identifies out-of-domain patients. These results suggest that Bayesian predictive uncertainty can greatly improve trustworthiness of machine learning models in high-risk settings such as the ICU.