AP-OOD: Attention Pooling for Out-of-Distribution Detection
This addresses the critical need for reliable deployment of machine learning models by improving OOD detection for text, representing a strong specific gain rather than a foundational advancement.
The paper tackled the problem of out-of-distribution detection in natural language by proposing AP-OOD, a method that uses attention pooling to aggregate token embeddings, resulting in state-of-the-art performance with reductions in FPR95 from 27.84% to 4.67% on XSUM and from 77.08% to 70.37% on WMT15.
Out-of-distribution (OOD) detection, which maps high-dimensional data into a scalar OOD score, is critical for the reliable deployment of machine learning models. A key challenge in recent research is how to effectively leverage and aggregate token embeddings from language models to obtain the OOD score. In this work, we propose AP-OOD, a novel OOD detection method for natural language that goes beyond simple average-based aggregation by exploiting token-level information. AP-OOD is a semi-supervised approach that flexibly interpolates between unsupervised and supervised settings, enabling the use of limited auxiliary outlier data. Empirically, AP-OOD sets a new state of the art in OOD detection for text: in the unsupervised setting, it reduces the FPR95 (false positive rate at 95% true positives) from 27.84% to 4.67% on XSUM summarization, and from 77.08% to 70.37% on WMT15 En-Fr translation.