Exploring Simple, High Quality Out-of-Distribution Detection with L2 Normalization
This provides a simple, efficient solution for out-of-distribution detection in machine learning, though it is incremental as it builds on existing normalization techniques.
The paper tackles out-of-distribution detection by showing that L2 normalization in feature space achieves competitive performance with minimal complexity, requiring only two lines of code and no extra tuning, and achieves results within two percentage points of state-of-the-art methods on some datasets.
We demonstrate that L2 normalization over feature space can produce capable performance for Out-of-Distribution (OoD) detection for some models and datasets. Although it does not demonstrate outright state-of-the-art performance, this method is notable for its extreme simplicity: it requires only two addition lines of code, and does not need specialized loss functions, image augmentations, outlier exposure or extra parameter tuning. We also observe that training may be more efficient for some datasets and architectures. Notably, only 60 epochs with ResNet18 on CIFAR10 (or 100 epochs with ResNet50) can produce performance within two percentage points (AUROC) of several state-of-the-art methods for some near and far OoD datasets. We provide theoretical and empirical support for this method, and demonstrate viability across five architectures and three In-Distribution (ID) datasets.