FRODO: Free rejection of out-of-distribution samples: application to chest x-ray analysis
This addresses the challenge of handling incorrect or irrelevant medical images for clinicians, though it is incremental as it builds on existing network architectures and metrics.
The paper tackled the problem of rejecting out-of-distribution samples in chest x-ray analysis by proposing a method that uses feature activations and a novel metric, achieving an AUC score of 0.99 for classification.
In this work, we propose a method to reject out-of-distribution samples which can be adapted to any network architecture and requires no additional training data. Publicly available chest x-ray data (38,353 images) is used to train a standard ResNet-50 model to detect emphysema. Feature activations of intermediate layers are used as descriptors defining the training data distribution. A novel metric, FRODO, is measured by using the Mahalanobis distance of a new test sample to the training data distribution. The method is tested using a held-out test dataset of 21,176 chest x-rays (in-distribution) and a set of 14,821 out-of-distribution x-ray images of incorrect orientation or anatomy. In classifying test samples as in or out-of distribution, our method achieves an AUC score of 0.99.