Learning How to Mutate Source Code from Bug-Fixes
This addresses the need for better mutation testing tools in software engineering, offering an automated method to improve test suite assessment, though it is incremental in applying deep learning to a known bottleneck.
The paper tackles the problem of generating realistic mutation operators for software testing by learning from real bug fixes, achieving a prediction accuracy of 9% to 45% for mutants resembling actual bugs and over 98% syntactic correctness.
Mutation testing has been widely accepted as an approach to guide test case generation or to assess the effectiveness of test suites. Empirical studies have shown that mutants are representative of real faults; yet they also indicated a clear need for better, possibly customized, mutation operators and strategies. While methods to devise domain-specific or general-purpose mutation operators from real faults exist, they are effort- and error-prone, and do not help the tester to decide whether and how to mutate a given source code element. We propose a novel approach to automatically learn mutants from faults in real programs. First, our approach processes bug fixing changes using fine-grained differencing, code abstraction, and change clustering. Then, it learns mutation models using a deep learning strategy. We have trained and evaluated our technique on a set of ~787k bug fixes mined from GitHub. Our empirical evaluation showed that our models are able to predict mutants that resemble the actual fixed bugs in between 9% and 45% of the cases, and over 98% of the automatically generated mutants are lexically and syntactically correct.