Antiforensic techniques deployed by custom developed malware in evading anti-virus detection
This work addresses malware evasion for cybersecurity practitioners, presenting incremental improvements to existing anti-forensic methods.
This research analyzed multiple anti-forensic techniques used by custom malware to evade antivirus detection, finding that specific evasion methods achieved varying success rates against different detection engines.
Both malware and antivirus detection tools advance in their capabilities. Malware aim is to evade the detection while antivirus is to detect the malware. Over time, the detection techniques evolved from simple static signature matching over antiheuristic analysis to machine learning assisted algorithms. This thesis describes several layers of anti-virus evasion deployed by the malware and conducts the analysis of the evasion success rate. The scientific contribution of this research is in the following techniques the malware used -- the new algorithm for identifying the Windows operating system functions, a new custom developed obfuscation and de-obfuscation routine and the usage of USB and sound devices enumeration in the anti-heuristic detection. The new PE mutation engine facilitates the malware static signature variation. In the next stage of the assessment, anti-virus engines then test the malware evasion capabilities. The locally installed antivirus applications and the two multi-scanner online engines inspect the submitted malware samples. The thesis examines the results and discusses the strengths and weaknesses of each evasion technique.