Outlier Detection by Consistent Data Selection Method
This work addresses outlier detection for applications like fraud and spam detection, but it is incremental as it builds on existing unsupervised clustering and one-class classification techniques.
The paper tackles the problem of outlier detection in scenarios with limited and evolving patterns, such as fraud and spam detection, by proposing a two-phase method that first retrieves consistent non-outlier data points and then uses a one-class classifier to identify outliers, achieving competitive performance on three public datasets.
Often the challenge associated with tasks like fraud and spam detection[1] is the lack of all likely patterns needed to train suitable supervised learning models. In order to overcome this limitation, such tasks are attempted as outlier or anomaly detection tasks. We also hypothesize that out- liers have behavioral patterns that change over time. Limited data and continuously changing patterns makes learning significantly difficult. In this work we are proposing an approach that detects outliers in large data sets by relying on data points that are consistent. The primary contribution of this work is that it will quickly help retrieve samples for both consistent and non-outlier data sets and is also mindful of new outlier patterns. No prior knowledge of each set is required to extract the samples. The method consists of two phases, in the first phase, consistent data points (non- outliers) are retrieved by an ensemble method of unsupervised clustering techniques and in the second phase a one class classifier trained on the consistent data point set is ap- plied on the remaining sample set to identify the outliers. The approach is tested on three publicly available data sets and the performance scores are competitive.