A Deep Learning Anomaly Detection Method in Textual Data
This work addresses anomaly detection in textual data, which is an incremental improvement over existing methods.
The authors tackled text anomaly detection by combining deep learning transformers with classical machine learning algorithms, reporting that their approach could potentially reduce false positive rates compared to other tested methods.
In this article, we propose using deep learning and transformer architectures combined with classical machine learning algorithms to detect and identify text anomalies in texts. Deep learning model provides a very crucial context information about the textual data which all textual context are converted to a numerical representation. We used multiple machine learning methods such as Sentence Transformers, Auto Encoders, Logistic Regression and Distance calculation methods to predict anomalies. The method are tested on the texts data and we used syntactic data from different source injected into the original text as anomalies or use them as target. Different methods and algorithm are explained in the field of outlier detection and the results of the best technique is presented. These results suggest that our algorithm could potentially reduce false positive rates compared with other anomaly detection methods that we are testing.