Exploring the Link Between Test Suite Quality and Automatic Specification Inference
This addresses the challenge of incomplete or incorrect specifications in industrial software systems, though it is incremental as it builds on existing invariant extraction tools.
The paper tackled the problem of automatically inferring specifications from test suites to aid software engineers, finding that instruction, branch, and method coverage correlate with high recall values up to 97.93%.
While no one doubts the importance of correct and complete specifications, many industrial systems still do not have formal specifications written out -- and even when they do, it is hard to check their correctness and completeness. This work explores the possibility of using an invariant extraction tool such as Daikon to automatically infer specifications from available test suites with the idea of aiding software engineers to improve the specifications by having another version to compare to. Given that our initial experiments did not produce satisfactory results, in this paper we explore which test suite attributes influence the quality of the inferred specification. Following further study, we found that instruction, branch and method coverage are correlated to high recall values, reaching up to 97.93%.