Deep Learning for Anomaly Detection in Log Data: A Survey
It addresses the problem of early incident detection for system operators by summarizing existing techniques, but is incremental as it reviews rather than proposes new methods.
This survey provides an overview of deep learning approaches for anomaly detection in log data, highlighting their superior performance over conventional methods and ability to handle unstable data formats, but does not include quantitative comparisons.
Automatic log file analysis enables early detection of relevant incidents such as system failures. In particular, self-learning anomaly detection techniques capture patterns in log data and subsequently report unexpected log event occurrences to system operators without the need to provide or manually model anomalous scenarios in advance. Recently, an increasing number of approaches leveraging deep learning neural networks for this purpose have been presented. These approaches have demonstrated superior detection performance in comparison to conventional machine learning techniques and simultaneously resolve issues with unstable data formats. However, there exist many different architectures for deep learning and it is non-trivial to encode raw and unstructured log data to be analyzed by neural networks. We therefore carry out a systematic literature review that provides an overview of deployed models, data pre-processing mechanisms, anomaly detection techniques, and evaluations. The survey does not quantitatively compare existing approaches but instead aims to help readers understand relevant aspects of different model architectures and emphasizes open issues for future work.