LogFormer: A Pre-train and Tuning Pipeline for Log Anomaly Detection
This work addresses the inefficiency of retraining for multi-domain log anomaly detection in AIOps, offering a more generalizable solution for industrial scenarios.
The paper tackles the problem of poor generalization in log anomaly detection across different domains by proposing LogFormer, a two-stage Transformer-based framework with pre-training and adapter-based tuning, which achieves effective results on multiple benchmarks with fewer trainable parameters and lower training costs.
Log anomaly detection is a key component in the field of artificial intelligence for IT operations (AIOps). Considering log data of variant domains, retraining the whole network for unknown domains is inefficient in real industrial scenarios. However, previous deep models merely focused on extracting the semantics of log sequences in the same domain, leading to poor generalization on multi-domain logs. To alleviate this issue, we propose a unified Transformer-based framework for Log anomaly detection (LogFormer) to improve the generalization ability across different domains, where we establish a two-stage process including the pre-training and adapter-based tuning stage. Specifically, our model is first pre-trained on the source domain to obtain shared semantic knowledge of log data. Then, we transfer such knowledge to the target domain via shared parameters. Besides, the Log-Attention module is proposed to supplement the information ignored by the log-paring. The proposed method is evaluated on three public and one real-world datasets. Experimental results on multiple benchmarks demonstrate the effectiveness of our LogFormer with fewer trainable parameters and lower training costs.