Cluster Purging: Efficient Outlier Detection based on Rate-Distortion Theory
This addresses outlier detection for data analysis, offering an incremental improvement with efficient algorithms.
The paper tackles outlier detection by extending clustering-based methods with rate-distortion theory, proposing two efficient algorithms (one parameter-free) that improve upon raw clustering and compete strongly against state-of-the-art alternatives.
Rate-distortion theory-based outlier detection builds upon the rationale that a good data compression will encode outliers with unique symbols. Based on this rationale, we propose Cluster Purging, which is an extension of clustering-based outlier detection. This extension allows one to assess the representivity of clusterings, and to find data that are best represented by individual unique clusters. We propose two efficient algorithms for performing Cluster Purging, one being parameter-free, while the other algorithm has a parameter that controls representivity estimations, allowing it to be tuned in supervised setups. In an experimental evaluation, we show that Cluster Purging improves upon outliers detected from raw clusterings, and that Cluster Purging competes strongly against state-of-the-art alternatives.