Dividing Deep Learning Model for Continuous Anomaly Detection of Inconsistent ICT Systems
This addresses the issue of maintaining reliable health monitoring in ICT systems when equipment replacements cause log data changes, though it is incremental as it builds on existing deep learning-based anomaly detection methods.
The paper tackles the problem of anomaly detection in ICT systems when log data changes after training, by proposing a method that divides deep learning models based on log data correlations to maintain monitoring without accuracy loss. Experiments on benchmark and real data show that the method does not decrease anomaly detection accuracy and allows continuous monitoring despite log data changes.
Health monitoring is important for maintaining reliable information and communications technology (ICT) systems. Anomaly detection methods based on machine learning, which train a model for describing "normality" are promising for monitoring the state of ICT systems. However, these methods cannot be used when the type of monitored log data changes from that of training data due to the replacement of certain equipment. Therefore, such methods may dismiss an anomaly that appears when log data changes. To solve this problem, we propose an ICT-systems-monitoring method with deep learning models divided based on the correlation of log data. We also propose an algorithm for extracting the correlations of log data from a deep learning model and separating log data based on the correlation. When some of the log data changes, our method can continue health monitoring with the divided models which are not affected by changes in the log data. We present the results from experiments involving benchmark data and real log data, which indicate that our method using divided models does not decrease anomaly detection accuracy and a model for anomaly detection can be divided to continue monitoring a network state even if some the log data change.