Mehrdad Abdi

Ebert Schoofs, Mehrdad Abdi, Serge Demeyer

Test Amplification is a method to extend handwritten tests into a more rigorous test suite covering corner cases in the system under test. Unfortunately, the current state-of-the-art for test amplification heavily relies on program analysis techniques which benefit a lot from explicit type declarations present in statically typed languages like Java and C++. In dynamically typed languages, such type declarations are not available and as a consequence test amplification has yet to find its way to programming languages like Python, Ruby and Javascript. In this paper, we present AmPyfier, a proof-of-concept tool, which brings test amplification to the dynamically typed, interpreted language Python. We evaluated this approach on 7 open-source projects, and found that AmPyfier could successfully strengthen 7 out of 10 test classes (70%). As such we demonstrate that test amplification is feasible for one of the most popular programming languages in use today.

Mehrdad Abdi, Henrique Rocha, Serge Demeyer et al.

Some test amplification tools extend a manually created test suite with additional test cases to increase the code coverage. The technique is effective, in the sense that it suggests strong and understandable test cases, generally adopted by software engineers. Unfortunately, the current state-of-the-art for test amplification heavily relies on program analysis techniques which benefit a lot from explicit type declarations present in statically typed languages. In dynamically typed languages, such type declarations are not available and as a consequence test amplification has yet to find its way to programming languages like Smalltalk, Python, Ruby and Javascript. We propose to exploit profiling information --readily obtainable by executing the associated test suite-- to infer the necessary type information creating special test inputs with corresponding assertions. We evaluated this approach on 52 selected test classes from 13 mature projects in the Pharo ecosystem containing approximately 400 test methods. We show the improvement in killing new mutants and mutation coverage at least in 28 out of 52 test classes (53%). Moreover, these generated tests are understandable by humans: 8 out of 11 pull-requests submitted were merged into the main code base (72%). These results are comparable to the state-of-the-art, hence we conclude that test amplification is feasible for dynamically typed languages.