Improving Scalability of Contrast Pattern Mining for Network Traffic Using Closed Patterns
This work addresses scalability issues in contrast pattern mining for network traffic analysis, which is incremental as it builds on existing methods to improve efficiency.
The paper tackled the challenge of efficiently finding relevant contrast patterns in high-dimensional datasets, specifically for network traffic analysis, by using closed patterns to reduce redundancy, resulting in an algorithm that is up to 100 times faster than an existing approach.
Contrast pattern mining (CPM) aims to discover patterns whose support increases significantly from a background dataset compared to a target dataset. CPM is particularly useful for characterising changes in evolving systems, e.g., in network traffic analysis to detect unusual activity. While most existing techniques focus on extracting either the whole set of contrast patterns (CPs) or minimal sets, the problem of efficiently finding a relevant subset of CPs, especially in high dimensional datasets, is an open challenge. In this paper, we focus on extracting the most specific set of CPs to discover significant changes between two datasets. Our approach to this problem uses closed patterns to substantially reduce redundant patterns. Our experimental results on several real and emulated network traffic datasets demonstrate that our proposed unsupervised algorithm is up to 100 times faster than an existing approach for CPM on network traffic data [2]. In addition, as an application of CPs, we demonstrate that CPM is a highly effective method for detection of meaningful changes in network traffic.