Text Anomaly Detection with Simplified Isolation Kernel
This work addresses efficiency issues in text anomaly detection for applications requiring low resource usage, though it is incremental as it builds on existing two-step approaches.
The paper tackles the challenge of high memory and computation costs in text anomaly detection by introducing the Simplified Isolation Kernel (SIK), which maps high-dimensional embeddings to sparse representations, achieving better detection performance than 11 SOTA algorithms across 7 datasets.
Two-step approaches combining pre-trained large language model embeddings and anomaly detectors demonstrate strong performance in text anomaly detection by leveraging rich semantic representations. However, high-dimensional dense embeddings extracted by large language models pose challenges due to substantial memory requirements and high computation time. To address this challenge, we introduce the Simplified Isolation Kernel (SIK), which maps high-dimensional dense embeddings to lower-dimensional sparse representations while preserving crucial anomaly characteristics. SIK has linear time complexity and significantly reduces space complexity through its innovative boundary-focused feature mapping. Experiments across 7 datasets demonstrate that SIK achieves better detection performance than 11 state-of-the-art (SOTA) anomaly detection algorithms while maintaining computational efficiency and low memory cost. All code and demonstrations are available at https://github.com/charles-cao/SIK.