SDOoop: Capturing Periodical Patterns and Out-of-phase Anomalies in Streaming Data Analysis
This work addresses the need for interpretable and efficient anomaly detection in streaming applications like IoT and cybersecurity, though it is incremental as it builds on an existing method.
The authors tackled the problem of detecting contextual anomalies and temporal patterns in streaming data by extending the SDO method to retain temporal information, resulting in SDOoop achieving equivalent or superior performance to state-of-the-art approaches in intrusion detection and natural science domains.
Streaming data analysis is increasingly required in applications, e.g., IoT, cybersecurity, robotics, mechatronics or cyber-physical systems. Despite its relevance, it is still an emerging field with open challenges. SDO is a recent anomaly detection method designed to meet requirements of speed, interpretability and intuitive parameterization. In this work, we present SDOoop, which extends the capabilities of SDO's streaming version to retain temporal information of data structures. SDOoop spots contextual anomalies undetectable by traditional algorithms, while enabling the inspection of data geometries, clusters and temporal patterns. We used SDOoop to model real network communications in critical infrastructures and extract patterns that disclose their dynamics. Moreover, we evaluated SDOoop with data from intrusion detection and natural science domains and obtained performances equivalent or superior to state-of-the-art approaches. Our results show the high potential of new model-based methods to analyze and explain streaming data. Since SDOoop operates with constant per-sample space and time complexity, it is ideal for big data, being able to instantly process large volumes of information. SDOoop conforms to next-generation machine learning, which, in addition to accuracy and speed, is expected to provide highly interpretable and informative models.