Unsupervised anomaly detection algorithms on real-world data: how many do we need?
This work provides a comprehensive benchmark for practitioners in anomaly detection, though it is incremental as it focuses on comparing existing methods on new datasets.
The study evaluated 32 unsupervised anomaly detection algorithms on 52 real-world multivariate tabular datasets, finding that k-thNN significantly outperformed most others overall, with kNN best on local datasets and EIF best on global datasets, and concluded that a toolbox of these three algorithms suffices for this data.
In this study we evaluate 32 unsupervised anomaly detection algorithms on 52 real-world multivariate tabular datasets, performing the largest comparison of unsupervised anomaly detection algorithms to date. On this collection of datasets, the $k$-thNN (distance to the $k$-nearest neighbor) algorithm significantly outperforms the most other algorithms. Visualizing and then clustering the relative performance of the considered algorithms on all datasets, we identify two clear clusters: one with ``local'' datasets, and another with ``global'' datasets. ``Local'' anomalies occupy a region with low density when compared to nearby samples, while ``global'' occupy an overall low density region in the feature space. On the local datasets the $k$NN ($k$-nearest neighbor) algorithm comes out on top. On the global datasets, the EIF (extended isolation forest) algorithm performs the best. Also taking into consideration the algorithms' computational complexity, a toolbox with these three unsupervised anomaly detection algorithms suffices for finding anomalies in this representative collection of multivariate datasets. By providing access to code and datasets, our study can be easily reproduced and extended with more algorithms and/or datasets.