Continuously Testing Distributed IoT Systems: An Overview of the State of the Art
This work addresses the need for effective testing methods in IoT development, which is crucial for developers and engineers, but it is incremental as it synthesizes existing research rather than introducing new methods.
This paper tackles the problem of continuous testing for distributed IoT systems by providing an overview of state-of-the-art tools and approaches, identifying challenges such as scale and heterogeneity, and comparing 18 tools to derive desirable characteristics for IoT test environments.
The continuous testing of small changes to systems has proven to be useful and is widely adopted in the development of software systems. For this, software is tested in environments that are as close as possible to the production environments. When testing IoT systems, this approach is met with unique challenges that stem from the typically large scale of the deployments, heterogeneity of nodes, challenging network characteristics, and tight integration with the environment among others. IoT test environments present a possible solution to these challenges by emulating the nodes, networks, and possibly domain environments in which IoT applications can be executed. This paper gives an overview of the state of the art in IoT testing. We derive desirable characteristics of IoT test environments, compare 18 tools that can be used in this respect, and give a research outlook of future trends in this area.