LogAction: Consistent Cross-system Anomaly Detection through Logs via Active Domain Adaptation
This work addresses the problem of efficient anomaly detection for software system reliability, offering a domain-specific solution that is incremental in combining existing techniques.
The paper tackles the challenge of log-based anomaly detection by proposing LogAction, which integrates active learning and domain adaptation to reduce labeling effort while addressing data distribution gaps and cold-start issues. It achieves an average 93.01% F1 score with only 2% manual labels, outperforming state-of-the-art methods by 26.28%.
Log-based anomaly detection is a essential task for ensuring the reliability and performance of software systems. However, the performance of existing anomaly detection methods heavily relies on labeling, while labeling a large volume of logs is highly challenging. To address this issue, many approaches based on transfer learning and active learning have been proposed. Nevertheless, their effectiveness is hindered by issues such as the gap between source and target system data distributions and cold-start problems. In this paper, we propose LogAction, a novel log-based anomaly detection model based on active domain adaptation. LogAction integrates transfer learning and active learning techniques. On one hand, it uses labeled data from a mature system to train a base model, mitigating the cold-start issue in active learning. On the other hand, LogAction utilize free energy-based sampling and uncertainty-based sampling to select logs located at the distribution boundaries for manual labeling, thus addresses the data distribution gap in transfer learning with minimal human labeling efforts. Experimental results on six different combinations of datasets demonstrate that LogAction achieves an average 93.01% F1 score with only 2% of manual labels, outperforming some state-of-the-art methods by 26.28%. Website: https://logaction.github.io