Beyond a Single Perspective: Text Anomaly Detection with Multi-View Language Representations
This work addresses the problem of improving adaptability and effectiveness in text anomaly detection for applications like content moderation and spam filtering, representing an incremental advance over existing methods.
The paper tackles the limitations of single-embedding models in text anomaly detection by proposing MCA^2, a multi-view framework that integrates embeddings from multiple pretrained language models, achieving state-of-the-art performance across 10 benchmark datasets.
Text anomaly detection (TAD) plays a critical role in various language-driven real-world applications, including harmful content moderation, phishing detection, and spam review filtering. While two-step "embedding-detector" TAD methods have shown state-of-the-art performance, their effectiveness is often limited by the use of a single embedding model and the lack of adaptability across diverse datasets and anomaly types. To address these limitations, we propose to exploit the embeddings from multiple pretrained language models and integrate them into $MCA^2$, a multi-view TAD framework. $MCA^2$ adopts a multi-view reconstruction model to effectively extract normal textual patterns from multiple embedding perspectives. To exploit inter-view complementarity, a contrastive collaboration module is designed to leverage and strengthen the interactions across different views. Moreover, an adaptive allocation module is developed to automatically assign the contribution weight of each view, thereby improving the adaptability to diverse datasets. Extensive experiments on 10 benchmark datasets verify the effectiveness of $MCA^2$ against strong baselines. The source code of $MCA^2$ is available at https://github.com/yankehan/MCA2.