On the use of Mahalanobis distance for out-of-distribution detection with neural networks for medical imaging
This work addresses the problem of reliable out-of-distribution detection for neural networks in medical imaging, which is crucial for clinical safety, but it is incremental as it refines existing distance-based methods.
The paper challenges the assumption that there is an optimal neural network layer for applying Mahalanobis distance for out-of-distribution detection, showing the optimal layer varies with OOD pattern type and that using multiple detectors improves robustness, validated on CheXpert chest X-rays with pacemakers and unseen sex as OOD cases.
Implementing neural networks for clinical use in medical applications necessitates the ability for the network to detect when input data differs significantly from the training data, with the aim of preventing unreliable predictions. The community has developed several methods for out-of-distribution (OOD) detection, within which distance-based approaches - such as Mahalanobis distance - have shown potential. This paper challenges the prevailing community understanding that there is an optimal layer, or combination of layers, of a neural network for applying Mahalanobis distance for detection of any OOD pattern. Using synthetic artefacts to emulate OOD patterns, this paper shows the optimum layer to apply Mahalanobis distance changes with the type of OOD pattern, showing there is no one-fits-all solution. This paper also shows that separating this OOD detector into multiple detectors at different depths of the network can enhance the robustness for detecting different OOD patterns. These insights were validated on real-world OOD tasks, training models on CheXpert chest X-rays with no support devices, then using scans with unseen pacemakers (we manually labelled 50% of CheXpert for this research) and unseen sex as OOD cases. The results inform best-practices for the use of Mahalanobis distance for OOD detection. The manually annotated pacemaker labels and the project's code are available at: https://github.com/HarryAnthony/Mahalanobis-OOD-detection.