Detection of Coincidentally Correct Test Cases through Random Forests
This work addresses a specific issue in automated debugging for software developers, presenting an incremental improvement in test case quality assessment.
The paper tackles the problem of coincidentally correct test cases degrading fault localization performance by proposing a Random Forests-based hybrid approach to identify and handle these mislabeled passing tests, reporting a cost-effective analysis of flipping or trimming them.
The performance of coverage-based fault localization greatly depends on the quality of test cases being executed. These test cases execute some lines of the given program and determine whether the underlying tests are passed or failed. In particular, some test cases may be well-behaved (i.e., passed) while executing faulty statements. These test cases, also known as coincidentally correct test cases, may negatively influence the performance of the spectra-based fault localization and thus be less helpful as a tool for the purpose of automated debugging. In other words, the involvement of these coincidentally correct test cases may introduce noises to the fault localization computation and thus cause in divergence of effectively localizing the location of possible bugs in the given code. In this paper, we propose a hybrid approach of ensemble learning combined with a supervised learning algorithm namely, Random Forests (RF) for the purpose of correctly identifying test cases that are mislabeled to be the passing test cases. A cost-effective analysis of flipping the test status or trimming (i.e., eliminating from the computation) the coincidental correct test cases is also reported.