Is Your Anomaly Detector Ready for Change? Adapting AIOps Solutions to the Real World
This work addresses the practical challenge of model drift in IT operations for AIOps practitioners, but it is incremental as it builds on existing maintenance techniques.
The paper tackles the problem of maintaining anomaly detection models in AIOps as operational data changes over time, analyzing model update frequencies and retraining approaches, and finds that a data change monitoring tool can effectively determine when retraining is needed.
Anomaly detection techniques are essential in automating the monitoring of IT systems and operations. These techniques imply that machine learning algorithms are trained on operational data corresponding to a specific period of time and that they are continuously evaluated on newly emerging data. Operational data is constantly changing over time, which affects the performance of deployed anomaly detection models. Therefore, continuous model maintenance is required to preserve the performance of anomaly detectors over time. In this work, we analyze two different anomaly detection model maintenance techniques in terms of the model update frequency, namely blind model retraining and informed model retraining. We further investigate the effects of updating the model by retraining it on all the available data (full-history approach) and only the newest data (sliding window approach). Moreover, we investigate whether a data change monitoring tool is capable of determining when the anomaly detection model needs to be updated through retraining.