R+R: Revisiting Static Feature-Based Android Malware Detection using Machine Learning
This work addresses reproducibility concerns for security researchers and practitioners in Android malware detection, though it is incremental as it refines existing methodologies rather than introducing a new paradigm.
The paper tackled the problem of reproducibility in static feature-based Android malware detection by systematically evaluating six ML models, finding that simpler tree-based methods like XGBoost consistently outperform complex neural networks, especially after removing dataset duplicates.
Static feature-based Android malware detection using machine learning (ML) remains critical due to its scalability and efficiency. However, existing approaches often overlook security-critical reproducibility concerns, such as dataset duplication, inadequate hyperparameter tuning, and variance from random initialization. This can significantly compromise the practical effectiveness of these systems. In this paper, we systematically investigate these challenges by proposing a more rigorous methodology for model selection and evaluation. Using two widely used datasets, Drebin and APIGraph, we evaluate six ML models of varying complexity under both offline and continuous active learning settings. Our analysis demonstrates that, contrary to popular belief, well-tuned, simpler models, particularly tree-based methods like XGBoost, consistently outperform more complex neural networks, especially when duplicates are removed. To promote transparency and reproducibility, we open-source our codebase, which is extensible for integrating new models and datasets, facilitating reproducible security research.