Detecting Localized Density Anomalies in Multivariate Data via Coin-Flip Statistics
It addresses anomaly detection for applications like system monitoring and scientific discovery, offering an unsupervised approach with scalability, but it appears incremental as it builds on existing statistical methods for density comparison.
The paper tackles the problem of detecting localized density anomalies in multivariate data by introducing EagleEye, a method that models neighbor sequences as coin-flip processes, and demonstrates its effectiveness by identifying anomalies in synthetic data, detecting particle decay events in 0.3% of Large Hadron Collider data, and uncovering new localized temperature changes.
Detecting localized density differences in multivariate data is a crucial task in computational science. Such anomalies can indicate a critical system failure, lead to a groundbreaking scientific discovery, or reveal unexpected changes in data distribution. We introduce EagleEye, an anomaly detection method to compare two multivariate datasets with the aim of identifying local density anomalies, namely over- or under-densities affecting only localised regions of the feature space. Anomalies are detected by modelling, for each point, the ordered sequence of its neighbours' membership label as a coin-flipping process and monitoring deviations from the expected behaviour of such process. A unique advantage of our method is its ability to provide an accurate, entirely unsupervised estimate of the local signal purity. We demonstrate its effectiveness through experiments on both synthetic and real-world datasets. In synthetic data, EagleEye accurately detects anomalies in multiple dimensions even when they affect a tiny fraction of the data. When applied to a challenging resonant anomaly detection benchmark task in simulated Large Hadron Collider data, EagleEye successfully identifies particle decay events present in just 0.3% of the dataset. In global temperature data, EagleEye uncovers previously unidentified, geographically localised changes in temperature fields that occurred in the most recent years. Thanks to its key advantages of conceptual simplicity, computational efficiency, trivial parallelisation, and scalability, EagleEye is widely applicable across many fields.