Interpretable Feature Learning in Multivariate Big Data Analysis for Network Monitoring
This work addresses the need for interpretable data models in network monitoring and troubleshooting, though it is incremental as it builds on an existing MBDA methodology.
The authors tackled the problem of interpretable feature learning for network monitoring by extending the Multivariate Big Data Analysis (MBDA) methodology to automatically derive features from massive datasets, enabling detection and diagnosis of network anomalies in real-world case studies like UGR'16 and Dartmouth'18.
There is an increasing interest in the development of new data-driven models useful to assess the performance of communication networks. For many applications, like network monitoring and troubleshooting, a data model is of little use if it cannot be interpreted by a human operator. In this paper, we present an extension of the Multivariate Big Data Analysis (MBDA) methodology, a recently proposed interpretable data analysis tool. In this extension, we propose a solution to the automatic derivation of features, a cornerstone step for the application of MBDA when the amount of data is massive. The resulting network monitoring approach allows us to detect and diagnose disparate network anomalies, with a data-analysis workflow that combines the advantages of interpretable and interactive models with the power of parallel processing. We apply the extended MBDA to two case studies: UGR'16, a benchmark flow-based real-traffic dataset for anomaly detection, and Dartmouth'18, the longest and largest Wi-Fi trace known to date.