Detecting Avalanche Effect in Adversarial Settings: Spotting the Encryption Loops in Ransomware
This work addresses the problem of reverse engineering ransomware for security analysts, providing a more robust detection method that is resilient to adversarial evasion.
The authors present a new approach to detect encryption loops in ransomware by checking for the avalanche effect itself, using a record-and-replay mechanism with the Shapiro-Wilk normality test. Their tool achieves 0.0% false negative rate and 1.1% false positive rate, and successfully analyzes all tested ransomware samples from ten families.
Spotting encryption loops in binary-only ransomware is a critical reverse engineering task. Since the existence of avalanche effect, an intrinsic characteristic of any secure encryption algorithms, is unavoidable during a victim data encryption attack, it is a very promising direction to spot encryption loops through avalanche effect detection. Unfortunately, no existing work in this direction ensures that the being-checked effect is the avalanche effect itself. Although CipherXRay is inspired by avalanche effect, it only checks whether a "ripple effect" (i.e., a necessary but non-sufficient condition) of avalanche effect exists, allowing a straightforward counterattack to succeed. In this work, we present a new approach that checks the avalanche effect itself. Because the detection is conducted in adversarial settings (e.g., the ransomware author may obfuscate the code), a viable approach must tolerate inaccurate input \& output identification and must be resilient to adversarial evasion. These challenges are addressed by a novel record-and-replay detection mechanism that takes advantage of the statistical guarantees provided by the Shapiro-Wilk normality test. The experimental results show that our approach achieves 0.0\% false negative rate and 1.1\% false positive rate. When our tool is employed to reverse engineer real-world ransomware samples, it succeeds in analyzing all the ransomware samples selected from ten representative families.