A Large-Scale Study of Call Graph-based Impact Prediction using Mutation Testing
This work addresses impact analysis for software engineers to optimize testing, but it is incremental as it evaluates existing call graph methods on new data.
The study tackled the problem of predicting software elements impacted by code changes by evaluating four call graph types using mutation testing on 10 Java projects with 17,000 mutants, finding that basic call graphs provide the best precision-recall trade-off despite more sophisticated graphs increasing completeness.
In software engineering, impact analysis involves predicting the software elements (e.g., modules, classes, methods) potentially impacted by a change in the source code. Impact analysis is required to optimize the testing effort. In this paper, we propose an evaluation technique to predict impact propagation. Based on 10 open-source Java projects and 5 classical mutation operators, we create 17,000 mutants and study how the error they introduce propagates. This evaluation technique enables us to analyze impact prediction based on four types of call graph. Our results show that graph sophistication increases the completeness of impact prediction. However, and surprisingly to us, the most basic call graph gives the best trade-off between precision and recall for impact prediction.