Deep Representation Learning with an Information-theoretic Loss
This addresses the challenge of detecting unseen classes in tasks like anomaly detection, though it appears incremental as it extends existing deep data description models.
The paper tackles the problem of anomaly and out-of-distribution detection in deep neural networks by proposing a representation learning method that increases inter-class distances and within-class similarity in the embedded space, resulting in improved segmentation of normal classes and better identification of out-of-distribution samples.
This paper proposes a deep representation learning using an information-theoretic loss with an aim to increase the inter-class distances as well as within-class similarity in the embedded space. Tasks such as anomaly and out-of-distribution detection, in which test samples comes from classes unseen in training, are problematic for deep neural networks. For such tasks, it is not sufficient to merely discriminate between known classes. Our intuition is to represent the known classes in compact and separated embedded regions in order to decrease the possibility of known and unseen classes overlapping in the embedded space. We derive a loss from Information Bottleneck principle, which reflects the inter-class distances as well as the compactness within classes, thus will extend the existing deep data description models. Our empirical study shows that the proposed model improves the segmentation of normal classes in the deep feature space, and subsequently contributes to identifying out-of-distribution samples.