Maat: Automatically Analyzing VirusTotal for Accurate Labeling and Effective Malware Detection
This work addresses a standardization issue for malware detection researchers who rely on VirusTotal, offering a more sustainable and accurate labeling method, though it is incremental as it builds on existing ML approaches.
The paper tackled the problem of inconsistent and unsustainable labeling of Android malware using VirusTotal scan results by introducing Maat, an automated machine learning-based labeling scheme that outperforms threshold-based strategies, achieving more accurate and consistent labeling and enabling more effective malware detection.
The malware analysis and detection research community relies on the online platform VirusTotal to label Android apps based on the scan results of around 60 antiviral scanners. Unfortunately, there are no standards on how to best interpret the scan results acquired from VirusTotal, which leads to the utilization of different threshold-based labeling strategies (e.g., if ten or more scanners deem an app malicious, it is considered malicious). While some of the utilized thresholds may be able to accurately approximate the ground truths of apps, the fact that VirusTotal changes the set and versions of the scanners it uses makes such thresholds unsustainable over time. We implemented a method, Maat, that tackles these issues of standardization and sustainability by automatically generating a Machine Learning (ML)-based labeling scheme, which outperforms threshold-based labeling strategies. Using the VirusTotal scan reports of 53K Android apps that span one year, we evaluated the applicability of Maat's ML-based labeling strategies by comparing their performance against threshold-based strategies. We found that such ML-based strategies (a) can accurately and consistently label apps based on their VirusTotal scan reports, and (b) contribute to training ML-based detection methods that are more effective at classifying out-of-sample apps than their threshold-based counterparts.