AnomalyBERT: Self-Supervised Transformer for Time Series Anomaly Detection using Data Degradation Scheme
This addresses the challenging problem of anomaly detection in unlabeled multivariate time series, such as sensor or network data, for applications like mechanical defect monitoring, but it is incremental as it builds on existing Transformer and self-supervised learning approaches.
The paper tackles the problem of detecting anomalies in unlabeled multivariate time series by proposing AnomalyBERT, a self-supervised Transformer model that uses a data degradation scheme with synthetic outliers. It surpasses previous state-of-the-art methods on five real-world benchmarks, demonstrating high efficiency in detecting anomalies in complex time series.
Mechanical defects in real situations affect observation values and cause abnormalities in multivariate time series, such as sensor values or network data. To perceive abnormalities in such data, it is crucial to understand the temporal context and interrelation between variables simultaneously. The anomaly detection task for time series, especially for unlabeled data, has been a challenging problem, and we address it by applying a suitable data degradation scheme to self-supervised model training. We define four types of synthetic outliers and propose the degradation scheme in which a portion of input data is replaced with one of the synthetic outliers. Inspired by the self-attention mechanism, we design a Transformer-based architecture to recognize the temporal context and detect unnatural sequences with high efficiency. Our model converts multivariate data points into temporal representations with relative position bias and yields anomaly scores from these representations. Our method, AnomalyBERT, shows a great capability of detecting anomalies contained in complex time series and surpasses previous state-of-the-art methods on five real-world benchmarks. Our code is available at https://github.com/Jhryu30/AnomalyBERT.