Assessing and Comparing Mutation-based Fault Localization Techniques
This work addresses the practicality of fault localization techniques for software developers by evaluating them on real-world faults, though it is incremental as it compares existing methods.
The paper assessed and compared two mutation-based fault localization methods, Metallaxis and MUSE, on real-world programs and faults, finding that they require 18% and 37% of statements to locate faults and identify 50% and 80% of faults when inspecting 10 and 25 statements.
Recent research demonstrated that mutation-based fault localization techniques are relatively accurate and practical. However, these methods have never been compared and have only been assessed with simple hand-seeded faults. Therefore, their actual practicality is questionable when it comes to real-wold faults. To deal with this limitation we asses and compare the two main mutation-based fault localization methods, named Metallaxis and MUSE, on a set of real-world programs and faults. Our results based on three typical evaluation metrics indicate that mutation-based fault localization methods are relatively accurate and provide relevant information to developers. Overall, our result indicate that Metallaxis and MUSE require 18% and 37% of the program statements to find the sought faults. Additionally, both methods locate 50% and 80% of the studied faults when developers inspect 10 and 25 statements.