AutoFlow: An Autoencoder-based Approach for IP Flow Record Compression with Minimal Impact on Traffic Classification
This work addresses storage and processing challenges in network monitoring, offering a solution for more efficient and scalable network management, though it is incremental as it builds on existing autoencoder techniques.
This paper tackles the problem of massive IP flow record storage and analysis by proposing an autoencoder-based compression method that reduces data size by 1.313x while maintaining 99.27% accuracy in traffic classification, compared to 99.77% with uncompressed data.
Network monitoring generates massive volumes of IP flow records, posing significant challenges for storage and analysis. This paper presents a novel deep learning-based approach to compressing these records using autoencoders, enabling direct analysis of compressed data without requiring decompression. Unlike traditional compression methods, our approach reduces data volume while retaining the utility of compressed data for downstream analysis tasks, including distinguishing modern application protocols and encrypted traffic from popular services. Through extensive experiments on a real-world network traffic dataset, we demonstrate that our autoencoder-based compression achieves a 1.313x reduction in data size while maintaining 99.27% accuracy in a multi-class traffic classification task, compared to 99.77% accuracy with uncompressed data. This marginal decrease in performance is offset by substantial gains in storage and processing efficiency. The implications of this work extend to more efficient network monitoring and scalable, real-time network management solutions.