Conformal Risk Control for Pulmonary Nodule Detection
This addresses the need for reliable and transparent decision support in safety-critical healthcare, though it is incremental as it applies an existing uncertainty quantification technique to a specific medical domain.
The paper tackled the problem of predictive uncertainty in AI-assisted pulmonary nodule detection for lung cancer screening by applying conformal risk control to an advanced detection model, achieving sensitivity competitive with individual radiologists with a slight increase in false positives.
Quantitative tools are increasingly appealing for decision support in healthcare, driven by the growing capabilities of advanced AI systems. However, understanding the predictive uncertainties surrounding a tool's output is crucial for decision-makers to ensure reliable and transparent decisions. In this paper, we present a case study on pulmonary nodule detection for lung cancer screening, enhancing an advanced detection model with an uncertainty quantification technique called conformal risk control (CRC). We demonstrate that prediction sets with conformal guarantees are attractive measures of predictive uncertainty in the safety-critical healthcare domain, allowing end-users to achieve arbitrary validity by trading off false positives and providing formal statistical guarantees on model performance. Among ground-truth nodules annotated by at least three radiologists, our model achieves a sensitivity that is competitive with that generally achieved by individual radiologists, with a slight increase in false positives. Furthermore, we illustrate the risks of using off-the-shelve prediction models when faced with ontological uncertainty, such as when radiologists disagree on what constitutes the ground truth on pulmonary nodules.