Software Test Automation Maturity -- A Survey of the State of the Practice
This survey provides insights into current test automation practices for software industry practitioners, but it is incremental as it reports on existing practices without introducing new methods.
The paper surveyed 151 practitioners from over 101 organizations to assess test automation maturity, finding that practices like automated test case percentage and Agile/DevOps use correlate with higher maturity levels, with roles such as QA engineers and consultants showing response variations.
The software industry has seen an increasing interest in test automation. In this paper, we present a test automation maturity survey serving as a self-assessment for practitioners. Based on responses of 151 practitioners coming from above 101 organizations in 25 countries, we make observations regarding the state of the practice of test automation maturity: a) The level of test automation maturity in different organizations is differentiated by the practices they adopt; b) Practitioner reported the quite diverse situation with respect to different practices, e.g., 85\% practitioners agreed that their test teams have enough test automation expertise and skills, while 47\% of practitioners admitted that there is lack of guidelines on designing and executing automated tests; c) Some practices are strongly correlated and/or closely clustered; d) The percentage of automated test cases and the use of Agile and/or DevOps development models are good indicators for a higher test automation maturity level; (e) The roles of practitioners may affect response variation, e.g., QA engineers give the most optimistic answers, consultants give the most pessimistic answers. Our results give an insight into present test automation processes and practices and indicate chances for further improvement in the present industry.