The Automotive Take on Continuous Experimentation: A Multiple Case Study
This addresses the problem of adapting software development methodologies for automotive companies working on self-driving systems, though it is incremental as it applies an existing method to a new domain.
The study explored the application of Continuous Experimentation (CE) in developing self-driving vehicles, finding that companies expect higher quality and faster roll-outs but face challenges like safety concerns and organizational structures.
Recently, an increasingly growing number of companies is focusing on achieving self-driving systems towards SAE level 3 and higher. Such systems will have much more complex capabilities than today's advanced driver assistance systems (ADAS) like adaptive cruise control and lane-keeping assistance. For complex software systems in the Web-application domain, the logical successor for Continuous Integration and Deployment (CI/CD) is known as Continuous Experimentation (CE), where product owners jointly with engineers systematically run A/B experiments on possible new features to get quantifiable data about a feature's adoption from the users. While this methodology is increasingly adopted in software-intensive companies, our study is set out to explore advantages and challenges when applying CE during the development and roll-out of functionalities required for self-driving vehicles. This paper reports about the design and results from a multiple case study that was conducted at four companies including two automotive OEMs with a long history of developing vehicles, a Tier-1 supplier, and a start-up company within the area of automated driving systems. Unanimously, all expect higher quality and fast roll-out cycles to the fleet; as major challenges, however, safety concerns next to organizational structures are mentioned.