Malware Traffic Classification: Evaluation of Algorithms and an Automated Ground-truth Generation Pipeline
This work addresses the problem of network security for organizations by providing an incremental improvement in malware detection through automated labeling and semi-supervised techniques.
The paper tackles the challenge of classifying encrypted malware traffic by developing a semi-supervised classification pipeline using observable metadata and proposes an automated ground-truth generation pipeline for evaluation. It explores various clustering approaches with diverse features, resulting in a method that can serve as a baseline for detection models.
Identifying threats in a network traffic flow which is encrypted is uniquely challenging. On one hand it is extremely difficult to simply decrypt the traffic due to modern encryption algorithms. On the other hand, passing such an encrypted stream through pattern matching algorithms is useless because encryption ensures there aren't any. Moreover, evaluating such models is also difficult due to lack of labeled benign and malware datasets. Other approaches have tried to tackle this problem by employing observable meta-data gathered from the flow. We try to augment this approach by extending it to a semi-supervised malware classification pipeline using these observable meta-data. To this end, we explore and test different kind of clustering approaches which make use of unique and diverse set of features extracted from this observable meta-data. We also, propose an automated packet data-labeling pipeline to generate ground-truth data which can serve as a base-line to evaluate the classifiers mentioned above in particular, or any other detection model in general.