Algorithmic Frameworks for the Detection of High Density Anomalies
This addresses the problem of reducing false positives in anomaly detection for large or noisy datasets, such as in misbehavior detection and data quality analysis, but it is incremental as it builds on existing anomaly detection methods.
The study tackled the problem of detecting high-density anomalies, which are deviant cases in normal data regions, by introducing non-parametric algorithmic frameworks for unsupervised detection. The Iterative Partial Push (IPP) framework achieved the best detection results compared to baseline algorithms.
This study explores the concept of high-density anomalies. As opposed to the traditional concept of anomalies as isolated occurrences, high-density anomalies are deviant cases positioned in the most normal regions of the data space. Such anomalies are relevant for various practical use cases, such as misbehavior detection and data quality analysis. Effective methods for identifying them are particularly important when analyzing very large or noisy sets, for which traditional anomaly detection algorithms will return many false positives. In order to be able to identify high-density anomalies, this study introduces several non-parametric algorithmic frameworks for unsupervised detection. These frameworks are able to leverage existing underlying anomaly detection algorithms and offer different solutions for the balancing problem inherent in this detection task. The frameworks are evaluated with both synthetic and real-world datasets, and are compared with existing baseline algorithms for detecting traditional anomalies. The Iterative Partial Push (IPP) framework proves to yield the best detection results.