Knowledge Distillation for Anomaly Detection
This addresses the deployment challenge for anomaly detection in resource-limited environments, but it is incremental as it adapts existing knowledge distillation techniques to this domain.
The paper tackles the problem of deploying large unsupervised anomaly detection models on resource-constrained devices by compressing them into smaller supervised models using knowledge distillation, achieving comparable performance while significantly reducing size and memory footprint.
Unsupervised deep learning techniques are widely used to identify anomalous behaviour. The performance of such methods is a product of the amount of training data and the model size. However, the size is often a limiting factor for the deployment on resource-constrained devices. We present a novel procedure based on knowledge distillation for compressing an unsupervised anomaly detection model into a supervised deployable one and we suggest a set of techniques to improve the detection sensitivity. Compressed models perform comparably to their larger counterparts while significantly reducing the size and memory footprint.