Monitoring machine learning (ML)-based risk prediction algorithms in the presence of confounding medical interventions
This addresses a critical issue for healthcare practitioners using ML models, where confounding interventions can mislead performance monitoring, though it is incremental as it builds on existing monitoring methods.
The paper tackles the problem of monitoring ML-based risk prediction algorithms in healthcare when medical interventions confound outcomes, showing that valid inference is possible under certain conditions. They develop a new CUSUM monitoring procedure and demonstrate its benefits through simulations and a real-world example, detecting calibration decay in a risk calculator during the COVID-19 pandemic.
Performance monitoring of machine learning (ML)-based risk prediction models in healthcare is complicated by the issue of confounding medical interventions (CMI): when an algorithm predicts a patient to be at high risk for an adverse event, clinicians are more likely to administer prophylactic treatment and alter the very target that the algorithm aims to predict. A simple approach is to ignore CMI and monitor only the untreated patients, whose outcomes remain unaltered. In general, ignoring CMI may inflate Type I error because (i) untreated patients disproportionally represent those with low predicted risk and (ii) evolution in both the model and clinician trust in the model can induce complex dependencies that violate standard assumptions. Nevertheless, we show that valid inference is still possible if one monitors conditional performance and if either conditional exchangeability or time-constant selection bias hold. Specifically, we develop a new score-based cumulative sum (CUSUM) monitoring procedure with dynamic control limits. Through simulations, we demonstrate the benefits of combining model updating with monitoring and investigate how over-trust in a prediction model may delay detection of performance deterioration. Finally, we illustrate how these monitoring methods can be used to detect calibration decay of an ML-based risk calculator for postoperative nausea and vomiting during the COVID-19 pandemic.