Understanding normalization in contrastive representation learning and out-of-distribution detection
This work addresses the challenge of underperformance in anomaly detection for datasets like aerial or microscopy images, offering a flexible solution that can be applied as outlier exposure or self-supervised learning.
The paper tackles the problem of improving out-of-distribution detection in contrastive representation learning by exploring the ℓ₂-norm of features and incorporating out-of-distribution data, achieving superior performance in various scenarios such as unimodal and multimodal settings with image datasets.
Contrastive representation learning has emerged as an outstanding approach for anomaly detection. In this work, we explore the $\ell_2$-norm of contrastive features and its applications in out-of-distribution detection. We propose a simple method based on contrastive learning, which incorporates out-of-distribution data by discriminating against normal samples in the contrastive layer space. Our approach can be applied flexibly as an outlier exposure (OE) approach, where the out-of-distribution data is a huge collective of random images, or as a fully self-supervised learning approach, where the out-of-distribution data is self-generated by applying distribution-shifting transformations. The ability to incorporate additional out-of-distribution samples enables a feasible solution for datasets where AD methods based on contrastive learning generally underperform, such as aerial images or microscopy images. Furthermore, the high-quality features learned through contrastive learning consistently enhance performance in OE scenarios, even when the available out-of-distribution dataset is not diverse enough. Our extensive experiments demonstrate the superiority of our proposed method under various scenarios, including unimodal and multimodal settings, with various image datasets.