Hierarchical Reference Sets for Robust Unsupervised Detection of Scattered and Clustered Outliers
This addresses the challenge of robust outlier detection in IoT systems, which is crucial for applications like anomaly event detection and clustering, but it appears incremental as it builds on existing graph-based methods for outlier detection.
The paper tackles the problem of detecting both scattered and clustered outliers in unsupervised IoT data analysis, where clustered outliers can be mistaken for normal behavior due to high local density, and proposes a novel outlier detection paradigm using graph structures to incorporate multi-perspective reference sets, achieving effective recognition as demonstrated through extensive experiments.
Most real-world IoT data analysis tasks, such as clustering and anomaly event detection, are unsupervised and highly susceptible to the presence of outliers. In addition to sporadic scattered outliers caused by factors such as faulty sensor readings, IoT systems often exhibit clustered outliers. These occur when multiple devices or nodes produce similar anomalous measurements, for instance, owing to localized interference, emerging security threats, or regional false alarms, forming micro-clusters. These clustered outliers can be easily mistaken for normal behavior because of their relatively high local density, thereby obscuring the detection of both scattered and contextual anomalies. To address this, we propose a novel outlier detection paradigm that leverages the natural neighboring relationships using graph structures. This facilitates multi-perspective anomaly evaluation by incorporating reference sets at both local and global scales derived from the graph. Our approach enables the effective recognition of scattered outliers without interference from clustered anomalies, whereas the graph structure simultaneously helps reflect and isolate clustered outlier groups. Extensive experiments, including comparative performance analysis, ablation studies, validation on downstream clustering tasks, and evaluation of hyperparameter sensitivity, demonstrate the efficacy of the proposed method. The source code is available at https://github.com/gordonlok/DROD.