Time-Series Anomaly Detection Service at Microsoft
This work provides an accurate, efficient, and general anomaly detection service for large companies like Microsoft to monitor applications and services in real time, though it is incremental in combining existing techniques.
The paper tackles time-series anomaly detection for monitoring metrics like Page Views and Revenue at Microsoft, proposing a novel algorithm combining Spectral Residual (SR) from visual saliency detection with Convolutional Neural Networks (CNN) to achieve superior results compared to state-of-the-art baselines on public and production datasets.
Large companies need to monitor various metrics (for example, Page Views and Revenue) of their applications and services in real time. At Microsoft, we develop a time-series anomaly detection service which helps customers to monitor the time-series continuously and alert for potential incidents on time. In this paper, we introduce the pipeline and algorithm of our anomaly detection service, which is designed to be accurate, efficient and general. The pipeline consists of three major modules, including data ingestion, experimentation platform and online compute. To tackle the problem of time-series anomaly detection, we propose a novel algorithm based on Spectral Residual (SR) and Convolutional Neural Network (CNN). Our work is the first attempt to borrow the SR model from visual saliency detection domain to time-series anomaly detection. Moreover, we innovatively combine SR and CNN together to improve the performance of SR model. Our approach achieves superior experimental results compared with state-of-the-art baselines on both public datasets and Microsoft production data.