Classifying Malware Using Function Representations in a Static Call Graph
This work provides a more principled, automated approach to malware family detection for cybersecurity analysts by reducing the need for manual feature engineering.
This paper proposes a deep learning method to classify malware families by analyzing static call graphs of x86 assembly instructions. Their system uses an RNN autoencoder to create dense vector representations of functions, which are then used as nodes in a graph to capture both local and global properties, achieving 99.41% classification accuracy on a Microsoft malware dataset.
We propose a deep learning approach for identifying malware families using the function call graphs of x86 assembly instructions. Though prior work on static call graph analysis exists, very little involves the application of modern, principled feature learning techniques to the problem. In this paper, we introduce a system utilizing an executable's function call graph where function representations are obtained by way of a recurrent neural network (RNN) autoencoder which maps sequences of x86 instructions into dense, latent vectors. These function embeddings are then modeled as vertices in a graph with edges indicating call dependencies. Capturing rich, node-level representations as well as global, topological properties of an executable file greatly improves malware family detection rates and contributes to a more principled approach to the problem in a way that deliberately avoids tedious feature engineering and domain expertise. We test our approach by performing several experiments on a Microsoft malware classification data set and achieve excellent separation between malware families with a classification accuracy of 99.41%.