Federico Giaimo

Federico Giaimo, Hugo Andrade, Christian Berger

Recently, an increasingly growing number of companies is focusing on achieving self-driving systems towards SAE level 3 and higher. Such systems will have much more complex capabilities than today's advanced driver assistance systems (ADAS) like adaptive cruise control and lane-keeping assistance. For complex software systems in the Web-application domain, the logical successor for Continuous Integration and Deployment (CI/CD) is known as Continuous Experimentation (CE), where product owners jointly with engineers systematically run A/B experiments on possible new features to get quantifiable data about a feature's adoption from the users. While this methodology is increasingly adopted in software-intensive companies, our study is set out to explore advantages and challenges when applying CE during the development and roll-out of functionalities required for self-driving vehicles. This paper reports about the design and results from a multiple case study that was conducted at four companies including two automotive OEMs with a long history of developing vehicles, a Tier-1 supplier, and a start-up company within the area of automated driving systems. Unanimously, all expect higher quality and fast roll-out cycles to the fleet; as major challenges, however, safety concerns next to organizational structures are mentioned.

Federico Giaimo, Hugo Andrade, Christian Berger

Context: New software development patterns are emerging aiming at accelerating the process of delivering value. One is Continuous Experimentation, which allows to systematically deploy and run instrumented software variants during development phase in order to collect data from the field of application. While currently this practice is used on a daily basis on web-based systems, technical difficulties challenge its adoption in fields where computational resources are constrained, e.g., cyber-physical systems and the automotive industry. Objective: This paper aims at providing an overview of the engagement on the Continuous Experimentation practice in the context of cyber-physical systems. %To provide an understanding of what is the state-of-the-art of the Continuous Experimentation practice in the context of cyber-physical systems, and what is the practitioners' feedback about this practice. Method: A systematic literature review has been conducted to investigate the link between the practice and the field of application. Additionally, an industrial multiple case study is reported. Results: The study presents the current state-of-the-art regarding Continuous Experimentation in the field of cyber-physical systems. The current perspective of Continuous Experimentation in industry is also reported. Conclusions: The field has not reached maturity yet. More conceptual analyses are found than solution proposals and the state-of-practice is yet to be achieved. However it is expected that in time an increasing number of solutions will be proposed and validated.

Federico Giaimo, Christian Berger

As the automotive industry focuses its attention more and more towards the software functionality of vehicles, techniques to deliver new software value at a fast pace are needed. Continuous Experimentation, a practice coming from the web-based systems world, is one of such techniques. It enables researchers and developers to use real-world data to verify their hypothesis and steer the software evolution based on performances and user preferences, reducing the reliance on simulations and guesswork. Several challenges prevent the verbatim adoption of this practice on automotive cyber-physical systems, e.g., safety concerns and limitations from computational resources; nonetheless, the automotive field is starting to take interest in this technique. This work aims at demonstrating and evaluating a prototypical Continuous Experimentation infrastructure, implemented on a distributed computational system housed in a commercial truck tractor that is used in daily operations by a logistic company on public roads. The system comprises computing units and sensors, and software deployment and data retrieval are only possible remotely via a mobile data connection due to the commercial interests of the logistics company. This study shows that the proposed experimentation process resulted in the development team being able to base software development choices on the real-world data collected during the experimental procedure. Additionally, a set of previously identified design criteria to enable Continuous Experimentation on automotive systems was discussed and their validity confirmed in the light of the presented work.

Federico Giaimo, Christian Berger, Crispin Kirchner

Autonomous vehicles are slowly becoming reality thanks to the efforts of many academic and industrial organizations. Due to the complexity of the software powering these systems and the dynamicity of the development processes, an architectural solution capable of supporting long-term evolution and maintenance is required. Continuous Experimentation (CE) is an already increasingly adopted practice in software-intensive web-based software systems to steadily improve them over time. CE allows organizations to steer the development efforts by basing decisions on data collected about the system in its field of application. Despite the advantages of Continuous Experimentation, this practice is only rarely adopted in cyber-physical systems and in the automotive domain. Reasons for this include the strict safety constraints and the computational capabilities needed from the target systems. In this work, a concept for using Continuous Experimentation for resource-constrained platforms like a self-driving vehicle is outlined.

Federico Giaimo, Christian Berger

The software powering today's vehicles surpasses mechatronics as the dominating engineering challenge due to its fast evolving and innovative nature. In addition, the software and system architecture for upcoming vehicles with automated driving functionality is already processing ~750MB/s - corresponding to over 180 simultaneous 4K-video streams from popular video-on-demand services. Hence, self-driving cars will run so much software to resemble "small data centers on wheels" rather than just transportation vehicles. Continuous Integration, Deployment, and Experimentation have been successfully adopted for software-only products as enabling methodology for feedback-based software development. For example, a popular search engine conducts ~250 experiments each day to improve the software based on its users' behavior. This work investigates design criteria for the software architecture and the corresponding software development and deployment process for complex cyber-physical systems, with the goal of enabling Continuous Experimentation as a way to achieve continuous software evolution. Our research involved reviewing related literature on the topic to extract relevant design requirements. The study is concluded by describing the software development and deployment process and software architecture adopted by our self-driving vehicle laboratory, both based on the extracted criteria.