Uncertainty estimation for out-of-distribution detection in computational histopathology
This addresses the safety issue for deploying AI in high-stakes clinical diagnoses by improving uncertainty estimation, though it is incremental as it builds on existing methods.
The paper tackled the problem of predictive uncertainty estimation for out-of-distribution detection in computational histopathology to enable safe clinical use, finding that a distance-aware method outperformed common approaches like Monte Carlo dropout and deep ensembles, but all methods showed a drop in performance and calibration on novel samples.
In computational histopathology algorithms now outperform humans on a range of tasks, but to date none are employed for automated diagnoses in the clinic. Before algorithms can be involved in such high-stakes decisions they need to "know when they don't know", i.e., they need to estimate their predictive uncertainty. This allows them to defer potentially erroneous predictions to a human pathologist, thus increasing their safety. Here, we evaluate the predictive performance and calibration of several uncertainty estimation methods on clinical histopathology data. We show that a distance-aware uncertainty estimation method outperforms commonly used approaches, such as Monte Carlo dropout and deep ensembles. However, we observe a drop in predictive performance and calibration on novel samples across all uncertainty estimation methods tested. We also investigate the use of uncertainty thresholding to reject out-of-distribution samples for selective prediction. We demonstrate the limitations of this approach and suggest areas for future research.