Semi-supervised novelty detection using ensembles with regularized disagreement
This work provides a more reliable method for identifying novel samples for expert evaluation, which is crucial for applications where unseen classes can lead to critical errors, such as in medical imaging.
This paper addresses the problem of deep neural networks making high-confidence predictions on unseen classes by developing a semi-supervised novelty detection (SSND) method. The method leverages an ensemble-based procedure with early stopping regularization to achieve disagreement specifically on out-of-distribution (OOD) data, significantly outperforming state-of-the-art SSND methods on standard image datasets and medical image datasets.
Deep neural networks often predict samples with high confidence even when they come from unseen classes and should instead be flagged for expert evaluation. Current novelty detection algorithms cannot reliably identify such near OOD points unless they have access to labeled data that is similar to these novel samples. In this paper, we develop a new ensemble-based procedure for semi-supervised novelty detection (SSND) that successfully leverages a mixture of unlabeled ID and novel-class samples to achieve good detection performance. In particular, we show how to achieve disagreement only on OOD data using early stopping regularization. While we prove this fact for a simple data distribution, our extensive experiments suggest that it holds true for more complex scenarios: our approach significantly outperforms state-of-the-art SSND methods on standard image data sets (SVHN/CIFAR-10/CIFAR-100) and medical image data sets with only a negligible increase in computation cost.