ADMoE: Anomaly Detection with Mixture-of-Experts from Noisy Labels
This work addresses the cost issue of label acquisition in anomaly detection, particularly for domains like enterprise security, by enabling neural network-based methods to handle noisy labels, though it is incremental as it builds on existing mixture-of-experts architectures.
The paper tackles the problem of anomaly detection requiring expensive clean labels by proposing ADMoE, a framework that leverages cheaper noisy labels, achieving up to 34% performance improvement over not using it and outperforming 13 leading baselines.
Existing works on anomaly detection (AD) rely on clean labels from human annotators that are expensive to acquire in practice. In this work, we propose a method to leverage weak/noisy labels (e.g., risk scores generated by machine rules for detecting malware) that are cheaper to obtain for anomaly detection. Specifically, we propose ADMoE, the first framework for anomaly detection algorithms to learn from noisy labels. In a nutshell, ADMoE leverages mixture-of-experts (MoE) architecture to encourage specialized and scalable learning from multiple noisy sources. It captures the similarities among noisy labels by sharing most model parameters, while encouraging specialization by building "expert" sub-networks. To further juice out the signals from noisy labels, ADMoE uses them as input features to facilitate expert learning. Extensive results on eight datasets (including a proprietary enterprise security dataset) demonstrate the effectiveness of ADMoE, where it brings up to 34% performance improvement over not using it. Also, it outperforms a total of 13 leading baselines with equivalent network parameters and FLOPS. Notably, ADMoE is model-agnostic to enable any neural network-based detection methods to handle noisy labels, where we showcase its results on both multiple-layer perceptron (MLP) and the leading AD method DeepSAD.