A Framework for In-Vivo Testing of Mobile Applications
This addresses the issue of configuration-related bugs for mobile app developers, but it is incremental as it builds on existing in-vivo testing concepts.
The paper tackles the problem of mobile app failures in untested configurations by proposing a framework for in-vivo testing that triggers tests when new configurations are detected in the field, resulting in negligible monitoring overhead (0-6%) and an average test execution time of 3 seconds.
The ecosystem in which mobile applications run is highly heterogeneous and configurable. All layers upon which mobile apps are built offer wide possibilities of variations, from the device and the hardware, to the operating system and middleware, up to the user preferences and settings. Testing all possible configurations exhaustively, before releasing the app, is unaffordable. As a consequence, the app may exhibit different, including faulty, behaviours when executed in the field, under specific configurations. In this paper, we describe a framework that can be instantiated to support in-vivo testing of a mobile app. The framework monitors the configuration in the field and triggers in-vivo testing when an untested configuration is recognized. Experimental results show that the overhead introduced by monitoring is unnoticeable to negligible (i.e., 0-6%) depending on the device being used (high- vs. low-end). In-vivo test execution required on average 3s: if performed upon screen lock activation, it introduces just a slight delay before locking the device.