Adaptive Anomaly Detection in Evolving Network Environments
This addresses the challenge of costly manual labeling and clean data requirements for anomaly detection in dynamic networks, though it is incremental as it builds on existing supervised and unsupervised approaches.
The paper tackles the problem of distribution shift in deep learning anomaly detection for network data by introducing NetSight, a framework that adapts online without manual labeling, achieving up to 11.72% F1-score improvement over state-of-the-art methods.
Distribution shift, a change in the statistical properties of data over time, poses a critical challenge for deep learning anomaly detection systems. Existing anomaly detection systems often struggle to adapt to these shifts. Specifically, systems based on supervised learning require costly manual labeling, while those based on unsupervised learning rely on clean data, which is difficult to obtain, for shift adaptation. Both of these requirements are challenging to meet in practice. In this paper, we introduce NetSight, a framework for supervised anomaly detection in network data that continually detects and adapts to distribution shifts in an online manner. NetSight eliminates manual intervention through a novel pseudo-labeling technique and uses a knowledge distillation-based adaptation strategy to prevent catastrophic forgetting. Evaluated on three long-term network datasets, NetSight demonstrates superior adaptation performance compared to state-of-the-art methods that rely on manual labeling, achieving F1-score improvements of up to 11.72%. This proves its robustness and effectiveness in dynamic networks that experience distribution shifts over time.