Uncertainty-aware abstention in medical diagnosis based on medical texts
This work addresses the critical problem of unreliable AI diagnoses for medical professionals and patients, though it is incremental as it builds on existing selective prediction approaches.
The study tackled reliability in AI-assisted medical diagnosis by exploring uncertainty quantification for selective prediction across multiple medical text tasks, and introduced HUQ-2, a new state-of-the-art method that effectively captures uncertainty, improving reliability in applications like mortality prediction and medical code classification.
This study addresses the critical issue of reliability for AI-assisted medical diagnosis. We focus on the selection prediction approach that allows the diagnosis system to abstain from providing the decision if it is not confident in the diagnosis. Such selective prediction (or abstention) approaches are usually based on the modeling predictive uncertainty of machine learning models involved. This study explores uncertainty quantification in machine learning models for medical text analysis, addressing diverse tasks across multiple datasets. We focus on binary mortality prediction from textual data in MIMIC-III, multi-label medical code prediction using ICD-10 codes from MIMIC-IV, and multi-class classification with a private outpatient visits dataset. Additionally, we analyze mental health datasets targeting depression and anxiety detection, utilizing various text-based sources, such as essays, social media posts, and clinical descriptions. In addition to comparing uncertainty methods, we introduce HUQ-2, a new state-of-the-art method for enhancing reliability in selective prediction tasks. Our results provide a detailed comparison of uncertainty quantification methods. They demonstrate the effectiveness of HUQ-2 in capturing and evaluating uncertainty, paving the way for more reliable and interpretable applications in medical text analysis.