BED: Bi-Encoder-Based Detectors for Out-of-Distribution Detection
This addresses the problem of identifying unknown or anomalous text inputs for NLP applications, offering a simple and effective solution with potential real-world impact, though it appears incremental as it builds on existing feature extractors and OOD detection frameworks.
The paper tackles out-of-distribution (OOD) detection in NLP by proposing bi-encoder-based detectors, which outperform existing methods across multiple datasets, achieving superior detection performance as measured by metrics like F1-Score and AUROC.
This paper introduces a novel method leveraging bi-encoder-based detectors along with a comprehensive study comparing different out-of-distribution (OOD) detection methods in NLP using different feature extractors. The feature extraction stage employs popular methods such as Universal Sentence Encoder (USE), BERT, MPNET, and GLOVE to extract informative representations from textual data. The evaluation is conducted on several datasets, including CLINC150, ROSTD-Coarse, SNIPS, and YELLOW. Performance is assessed using metrics such as F1-Score, MCC, FPR@90, FPR@95, AUPR, an AUROC. The experimental results demonstrate that the proposed bi-encoder-based detectors outperform other methods, both those that require OOD labels in training and those that do not, across all datasets, showing great potential for OOD detection in NLP. The simplicity of the training process and the superior detection performance make them applicable to real-world scenarios. The presented methods and benchmarking metrics serve as a valuable resource for future research in OOD detection, enabling further advancements in this field. The code and implementation details can be found on our GitHub repository: https://github.com/yellowmessenger/ood-detection.