What Makes Agile Test Artifacts Useful? An Activity-Based Quality Model from a Practitioners' Perspective
This addresses a gap in Agile software testing practices for practitioners, but it is incremental as it builds on known challenges like language and traceability.
The study tackled the lack of empirical understanding of how Agile test artifacts are designed and what quality factors make them useful, by conducting an industrial survey with 18 practitioners to identify nine challenges and 16 quality factors for six test artifacts.
Background: The artifacts used in Agile software testing and the reasons why these artifacts are used are fairly well-understood. However, empirical research on how Agile test artifacts are eventually designed in practice and which quality factors make them useful for software testing remains sparse. Aims: Our objective is two-fold. First, we identify current challenges in using test artifacts to understand why certain quality factors are considered good or bad. Second, we build an Activity-Based Artifact Quality Model that describes what Agile test artifacts should look like. Method: We conduct an industrial survey with 18 practitioners from 12 companies operating in seven different domains. Results: Our analysis reveals nine challenges and 16 factors describing the quality of six test artifacts from the perspective of Agile testers. Interestingly, we observed mostly challenges regarding language and traceability, which are well-known to occur in non-Agile projects. Conclusions: Although Agile software testing is becoming the norm, we still have little confidence about general do's and don'ts going beyond conventional wisdom. This study is the first to distill a list of quality factors deemed important to what can be considered as useful test artifacts.