Continuous, Evolutionary and Large-Scale: A New Perspective for Automated Mobile App Testing
This is a perspective paper that identifies challenges and proposes a vision for automated mobile app testing, which is incremental as it builds on existing surveys and frameworks.
The paper tackles the lack of comprehensive automated testing solutions for mobile apps by surveying current tools and proposing a new framework based on Continuous, Evolutionary, and Large-scale principles, but does not report concrete results or numbers.
Mobile app development involves a unique set of challenges including device fragmentation and rapidly evolving platforms, making testing a difficult task. The design space for a comprehensive mobile testing strategy includes features, inputs, potential contextual app states, and large combinations of devices and underlying platforms. Therefore, automated testing is an essential activity of the development process. However, current state of the art of automated testing tools for mobile apps poses limitations that has driven a preference for manual testing in practice. As of today, there is no comprehensive automated solution for mobile testing that overcomes fundamental issues such as automated oracles, history awareness in test cases, or automated evolution of test cases. In this perspective paper we survey the current state of the art in terms of the frameworks, tools, and services available to developers to aid in mobile testing, highlighting present shortcomings. Next, we provide commentary on current key challenges that restrict the possibility of a comprehensive, effective, and practical automated testing solution. Finally, we offer our vision of a comprehensive mobile app testing framework, complete with research agenda, that is succinctly summarized along three principles: Continuous, Evolutionary and Large-scale (CEL).