Continuous Experimentation and the Cyber-Physical Systems challenge: An overview of the literature and the industrial perspective
It addresses the challenge of adopting Continuous Experimentation in resource-constrained domains like automotive systems, but is incremental as it reviews existing literature and industry perspectives.
This paper provides an overview of Continuous Experimentation in cyber-physical systems, finding that the field lacks maturity with more conceptual analyses than practical solutions and no concrete numerical results reported.
Context: New software development patterns are emerging aiming at accelerating the process of delivering value. One is Continuous Experimentation, which allows to systematically deploy and run instrumented software variants during development phase in order to collect data from the field of application. While currently this practice is used on a daily basis on web-based systems, technical difficulties challenge its adoption in fields where computational resources are constrained, e.g., cyber-physical systems and the automotive industry. Objective: This paper aims at providing an overview of the engagement on the Continuous Experimentation practice in the context of cyber-physical systems. %To provide an understanding of what is the state-of-the-art of the Continuous Experimentation practice in the context of cyber-physical systems, and what is the practitioners' feedback about this practice. Method: A systematic literature review has been conducted to investigate the link between the practice and the field of application. Additionally, an industrial multiple case study is reported. Results: The study presents the current state-of-the-art regarding Continuous Experimentation in the field of cyber-physical systems. The current perspective of Continuous Experimentation in industry is also reported. Conclusions: The field has not reached maturity yet. More conceptual analyses are found than solution proposals and the state-of-practice is yet to be achieved. However it is expected that in time an increasing number of solutions will be proposed and validated.