Autoencoder-based Anomaly Detection in Streaming Data with Incremental Learning and Concept Drift Adaptation
This addresses the problem of detecting anomalies in unlabelled, non-stationary streaming data for applications like monitoring systems, though it is incremental as it builds on existing autoencoder and drift detection techniques.
The paper tackles anomaly detection in streaming data with concept drift and class imbalance, proposing an autoencoder-based method that combines incremental learning and drift detection, which significantly outperforms existing methods in experiments.
In our digital universe nowadays, enormous amount of data are produced in a streaming manner in a variety of application areas. These data are often unlabelled. In this case, identifying infrequent events, such as anomalies, poses a great challenge. This problem becomes even more difficult in non-stationary environments, which can cause deterioration of the predictive performance of a model. To address the above challenges, the paper proposes an autoencoder-based incremental learning method with drift detection (strAEm++DD). Our proposed method strAEm++DD leverages on the advantages of both incremental learning and drift detection. We conduct an experimental study using real-world and synthetic datasets with severe or extreme class imbalance, and provide an empirical analysis of strAEm++DD. We further conduct a comparative study, showing that the proposed method significantly outperforms existing baseline and advanced methods.