Perils of Label Indeterminacy: A Case Study on Prediction of Neurological Recovery After Cardiac Arrest
This work addresses a critical issue in high-stakes healthcare AI, highlighting ethical implications for model evaluation and design, though it is incremental in focusing on a specific case study.
The paper tackles the problem of label indeterminacy in AI-assisted decision-making, where different ways of estimating unknown labels lead to models with similar performance on known data but drastically different predictions for unknown cases, as shown in a study on predicting neurological recovery after cardiac arrest.
The design of AI systems to assist human decision-making typically requires the availability of labels to train and evaluate supervised models. Frequently, however, these labels are unknown, and different ways of estimating them involve unverifiable assumptions or arbitrary choices. In this work, we introduce the concept of label indeterminacy and derive important implications in high-stakes AI-assisted decision-making. We present an empirical study in a healthcare context, focusing specifically on predicting the recovery of comatose patients after resuscitation from cardiac arrest. Our study shows that label indeterminacy can result in models that perform similarly when evaluated on patients with known labels, but vary drastically in their predictions for patients where labels are unknown. After demonstrating crucial ethical implications of label indeterminacy in this high-stakes context, we discuss takeaways for evaluation, reporting, and design.