SN4KE: Practical Mutation Testing at Binary Level
This work addresses the need for mutation testing in binary-only software, which is incremental as it builds on existing binary rewriting tools to enhance mutation analysis.
The paper tackled the problem of evaluating test suite adequacy for software only available in binary format by proposing a practical binary mutation analysis using binary rewriting and rich mutation operators. The result showed that richer operators generated more diverse mutants, leading to a higher mutation score for test harnesses, and reassembleable disassembly rewriting offered better scalability compared to lifting to an intermediate representation.
Mutation analysis is an effective technique to evaluate a test suite adequacy in terms of revealing unforeseen bugs in software. Traditional source- or IR-level mutation analysis is not applicable to the software only available in binary format. This paper proposes a practical binary mutation analysis via binary rewriting, along with a rich set of mutation operators to represent more realistic bugs. We implemented our approach using two state-of-the-art binary rewriting tools and evaluated its effectiveness and scalability by applying them to SPEC CPU benchmarks. Our analysis revealed that the richer mutation operators contribute to generating more diverse mutants, which, compared to previous works leads to a higher mutation score for the test harness. We also conclude that the reassembleable disassembly rewriting yields better scalability in comparison to lifting to an intermediate representation and performing a full translation.