Resilient Self-Debugging Software Protection
This addresses security for software developers against reverse engineering and tampering, but it is incremental as it builds upon existing self-debugging techniques.
The paper tackled vulnerabilities in self-debugging software protection by improving its resilience through reciprocal debugging and stealthier control transfers, resulting in a design that is significantly harder for attackers to defeat.
Debuggers are a popular reverse engineering and tampering tool. Self-debugging is an effective technique for applications to defend themselves against hostile debuggers. In penetration tests on state-of-the-art self-debugging, we observed several opportunities through which it could be attacked, however. We therefore improved upon the existing technique, making it more resilient by introducing reciprocal debugging and making the transfers of control between protected application and self-debugger more stealthy. This paper presents the improved self-debugging design, and details our research efforts into realizing reciprocal debugging. In our evaluation we show that the improved design is significantly harder for attackers to defeat.