Dual Representation Learning for Out-of-Distribution Detection
This addresses the issue of high-confidence mispredictions for out-of-distribution samples in classification tasks, which is crucial for reliable AI deployment, though it appears incremental as it builds on existing information bottleneck concepts.
The paper tackles the problem of out-of-distribution detection in deep neural networks by proposing Dual Representation Learning, which explores both strongly and weakly label-related information to distinguish in- and out-of-distribution samples, resulting in outperforming state-of-the-art methods.
To classify in-distribution samples, deep neural networks explore strongly label-related information and discard weakly label-related information according to the information bottleneck. Out-of-distribution samples drawn from distributions differing from that of in-distribution samples could be assigned with unexpected high-confidence predictions because they could obtain minimum strongly label-related information. To distinguish in- and out-of-distribution samples, Dual Representation Learning (DRL) makes out-of-distribution samples harder to have high-confidence predictions by exploring both strongly and weakly label-related information from in-distribution samples. For a pretrained network exploring strongly label-related information to learn label-discriminative representations, DRL trains its auxiliary network exploring the remaining weakly label-related information to learn distribution-discriminative representations. Specifically, for a label-discriminative representation, DRL constructs its complementary distribution-discriminative representation by integrating diverse representations less similar to the label-discriminative representation. Accordingly, DRL combines label- and distribution-discriminative representations to detect out-of-distribution samples. Experiments show that DRL outperforms the state-of-the-art methods for out-of-distribution detection.