Enhancing Out-of-Distribution Detection in Natural Language Understanding via Implicit Layer Ensemble
This addresses the reliability issue in NLP systems for users by enhancing OOD detection, though it appears incremental as it builds on existing contrastive learning and layer ensemble techniques.
The paper tackled the problem of out-of-distribution detection in natural language understanding by proposing a contrastive learning framework that leverages intermediate layer features, resulting in significantly improved effectiveness over existing methods as demonstrated in experiments on intent classification and OOD datasets.
Out-of-distribution (OOD) detection aims to discern outliers from the intended data distribution, which is crucial to maintaining high reliability and a good user experience. Most recent studies in OOD detection utilize the information from a single representation that resides in the penultimate layer to determine whether the input is anomalous or not. Although such a method is straightforward, the potential of diverse information in the intermediate layers is overlooked. In this paper, we propose a novel framework based on contrastive learning that encourages intermediate features to learn layer-specialized representations and assembles them implicitly into a single representation to absorb rich information in the pre-trained language model. Extensive experiments in various intent classification and OOD datasets demonstrate that our approach is significantly more effective than other works.