Tests as Maintainable Assets Via Auto-generated Spies: A case study involving the Scala collections library's Iterator trait
This is an incremental improvement for software developers, particularly in the Scala community, aiming to enhance test maintainability and readability.
The paper tackled the problem of testing stateful abstractions by comparing manually implemented test spies with auto-generated ones using mocking frameworks like Mockito, finding that auto-generated spies reduce test code complexity by over 70% in some cases.
In testing stateful abstractions, it is often necessary to record interactions, such as method invocations, and express assertions over these interactions. Following the Test Spy design pattern, we can reify such interactions programmatically through additional mutable state. Alternatively, a mocking framework, such as Mockito, can automatically generate test spies that allow us to record the interactions and express our expectations in a declarative domain-specific language. According to our study of the test code for Scala's Iterator trait, the latter approach can lead to a significant reduction of test code complexity in terms of metrics such as code size (in some cases over 70% smaller), cyclomatic complexity, and amount of additional mutable state required. In this tools paper, we argue that the resulting test code is not only more maintainable, readable, and intentional, but also a better stylistic match for the Scala community than manually implemented, explicitly stateful test spies.