Anomaly Detection in Image Datasets Using Convolutional Neural Networks, Center Loss, and Mahalanobis Distance
This work addresses the issue of filtering anomalous images for data processing pipelines, which is incremental as it builds on existing neural network and center loss techniques.
The paper tackles the problem of detecting poor-quality or irrelevant images in datasets by proposing methods for supervised and semi-supervised detection of out-of-distribution samples, using a neural network extended with center loss and Mahalanobis distance to simultaneously handle image classification and anomaly detection, with analysis on MNIST and ImageNet-30 datasets.
User activities generate a significant number of poor-quality or irrelevant images and data vectors that cannot be processed in the main data processing pipeline or included in the training dataset. Such samples can be found with manual analysis by an expert or with anomalous detection algorithms. There are several formal definitions for the anomaly samples. For neural networks, the anomalous is usually defined as out-of-distribution samples. This work proposes methods for supervised and semi-supervised detection of out-of-distribution samples in image datasets. Our approach extends a typical neural network that solves the image classification problem. Thus, one neural network after extension can solve image classification and anomalous detection problems simultaneously. Proposed methods are based on the center loss and its effect on a deep feature distribution in a last hidden layer of the neural network. This paper provides an analysis of the proposed methods for the LeNet and EfficientNet-B0 on the MNIST and ImageNet-30 datasets.