Detecting when pre-trained nnU-Net models fail silently for Covid-19 lung lesion segmentation
This addresses the lack of trust in deep learning models for clinical use by preventing silent failures, though it is incremental as it builds on existing segmentation pipelines.
The paper tackles the problem of silent failures in pre-trained nnU-Net models for Covid-19 lung lesion segmentation on out-of-distribution data by proposing a lightweight OOD detection method using Mahalanobis distance, which effectively detects incorrectly segmented samples.
Automatic segmentation of lung lesions in computer tomography has the potential to ease the burden of clinicians during the Covid-19 pandemic. Yet predictive deep learning models are not trusted in the clinical routine due to failing silently in out-of-distribution (OOD) data. We propose a lightweight OOD detection method that exploits the Mahalanobis distance in the feature space. The proposed approach can be seamlessly integrated into state-of-the-art segmentation pipelines without requiring changes in model architecture or training procedure, and can therefore be used to assess the suitability of pre-trained models to new data. We validate our method with a patch-based nnU-Net architecture trained with a multi-institutional dataset and find that it effectively detects samples that the model segments incorrectly.