Leveraging Intermediate Representations for Better Out-of-Distribution Detection
This addresses the need for reliable OoD detection in real-world applications to prevent unsafe decisions, but it is incremental as it builds on existing methods by focusing on intermediate layers.
The paper tackles the problem of improving Out-of-Distribution (OoD) detection in machine learning models by leveraging intermediate layer activations, showing that this approach enhances performance across multiple datasets.
In real-world applications, machine learning models must reliably detect Out-of-Distribution (OoD) samples to prevent unsafe decisions. Current OoD detection methods often rely on analyzing the logits or the embeddings of the penultimate layer of a neural network. However, little work has been conducted on the exploitation of the rich information encoded in intermediate layers. To address this, we analyze the discriminative power of intermediate layers and show that they can positively be used for OoD detection. Therefore, we propose to regularize intermediate layers with an energy-based contrastive loss, and by grouping multiple layers in a single aggregated response. We demonstrate that intermediate layer activations improves OoD detection performance by running a comprehensive evaluation across multiple datasets.