Coincident Learning for Unsupervised Anomaly Detection
This work addresses the problem of detecting anomalies in complex systems like industrial facilities and large-scale experiments, where labeled data is scarce, offering a novel approach for multi-modal tasks.
The paper tackles unsupervised anomaly detection in multi-modal tasks by introducing CoAD, a method that identifies anomalies based on coincident behavior across different feature slices, using a novel unsupervised metric derived from the Fβ statistic. It demonstrates results on synthetic and real-world datasets, including metal milling and particle accelerator data, with competitive performance compared to prior state-of-the-art methods.
Anomaly detection is an important task for complex systems (e.g., industrial facilities, manufacturing, large-scale science experiments), where failures in a sub-system can lead to low yield, faulty products, or even damage to components. While complex systems often have a wealth of data, labeled anomalies are typically rare (or even nonexistent) and expensive to acquire. Unsupervised approaches are therefore common and typically search for anomalies either by distance or density of examples in the input feature space (or some associated low-dimensional representation). This paper presents a novel approach called CoAD, which is specifically designed for multi-modal tasks and identifies anomalies based on \textit{coincident} behavior across two different slices of the feature space. We define an \textit{unsupervised} metric, $\hat{F}_β$, out of analogy to the supervised classification $F_β$ statistic. CoAD uses $\hat{F}_β$ to train an anomaly detection algorithm on \textit{unlabeled data}, based on the expectation that anomalous behavior in one feature slice is coincident with anomalous behavior in the other. The method is illustrated using a synthetic outlier data set and a MNIST-based image data set, and is compared to prior state-of-the-art on two real-world tasks: a metal milling data set and a data set from a particle accelerator.