Alternatives for Testing of Context-Aware Contemporary Software Systems in industrial settings: Results from a Rapid review
This addresses testing challenges for context-aware software systems in industrial settings, but is incremental as it reviews existing approaches without proposing new methods.
The study investigated how industry handles context variation in testing context-aware software systems, finding that current research focuses on generating test suites through behavior modeling and increased computing resources, but lacks solutions for managing context variation throughout the testing lifecycle.
Context: Context-aware contemporary software systems (CACSS) are mainstream. Furthermore, they present challenges for current engineering practices. These challenges are distinctively present when testing CACSS, as the variation of context deepens the limitations of available software testing practices and technologies. Objective: To understand how the industry deals with the variation of context when testing CACSS. Method: A Rapid Review was commissioned to uncover the necessary evidence to achieve the objectives. Results: Our results show that current research initiatives aim to generate or improve Test Suites that can deal with the variation of context and the sheer volume of test input possibilities. To achieve this, they mostly rely on modelling the systems' dynamic behavior and increasing computing resources to generate test inputs. We found no evidence of research results aiming at managing context variation through the testing lifecycle process. Conclusions: We discuss how the identified solutions are not ready for mainstream adoption. They are all domain-specific, and while the ideas and approaches can be reproduced in different settings, the technologies noon to be re-engineered and tailor to the specific CACSS.