Pasi Kuvaja

Alireza Haghighatkhah, Mika Mäntylä, Markku Oivo et al.

Two heuristics namely diversity-based (DBTP) and history-based test prioritization (HBTP) have been separately proposed in the literature. Yet, their combination has not been widely studied in continuous integration (CI) environments. The objective of this study is to catch regression faults earlier, allowing developers to integrate and verify their changes more frequently and continuously. To achieve this, we investigated six open-source projects, each of which included several builds over a large time period. Findings indicate that previous failure knowledge seems to have strong predictive power in CI environments and can be used to effectively prioritize tests. HBTP does not necessarily need to have large data, and its effectiveness improves to a certain degree with larger history interval. DBTP can be used effectively during the early stages, when no historical data is available, and also combined with HBTP to improve its effectiveness. Among the investigated techniques, we found that history-based diversity using NCD Multiset is superior in terms of effectiveness but comes with relatively higher overhead in terms of method execution time. Test prioritization in CI environments can be effectively performed with negligible investment using previous failure knowledge, and its effectiveness can be further improved by considering dissimilarities among the tests.

Alireza Haghighatkhah, Mika Mäntylä, Markku Oivo et al.

A classical heuristic in software testing is to reward diversity, which implies that a higher priority must be assigned to test cases that differ the most from those already prioritized. This approach is commonly known as similarity-based test prioritization (SBTP) and can be realized using a variety of techniques. The objective of our study is to investigate whether SBTP is more effective at finding defects than random permutation, as well as determine which SBTP implementations lead to better results. To achieve our objective, we implemented five different techniques from the literature and conducted an experiment using the defects4j dataset, which contains 395 real faults from six real-world open-source Java programs. Findings indicate that running the most dissimilar test cases early in the process is largely more effective than random permutation (Vargha-Delaney A [VDA]: 0.76-0.99 observed using normalized compression distance). No technique was found to be superior with respect to the effectiveness. Locality-sensitive hashing was, to a small extent, less effective than other SBTP techniques (VDA: 0.38 observed in comparison to normalized compression distance), but its speed largely outperformed the other techniques (i.e., it was approximately 5-111 times faster). Our results bring to mind the well-known adage, "don't put all your eggs in one basket". To effectively consume a limited testing budget, one should spread it evenly across different parts of the system by running the most dissimilar test cases early in the testing process.

Ahmad Banijamali, Pooyan Jamshidi, Pasi Kuvaja et al.

Connecting vehicles to cloud platforms has enabled innovative business scenarios while raising new quality concerns, such as reliability and scalability, which must be addressed by research. Cloud-native architectures based on microservices are a recent approach to enable continuous delivery and to improve service reliability and scalability. We propose an approach for restructuring cloud platform architectures in the automotive domain into a microservices architecture. To this end, we adopted and implemented microservices patterns from literature to design the cloud-native automotive architecture and conducted a laboratory experiment to evaluate the reliability and scalability of microservices in the context of a real-world project in the automotive domain called Eclipse Kuksa. Findings indicated that the proposed architecture could handle the continuous software delivery over-the-air by sending automatic control messages to a vehicular setting. Different patterns enabled us to make changes or interrupt services without extending the impact to others. The results of this study provide evidences that microservices are a potential design solution when dealing with service failures and high payload on cloud-based services in the automotive domain.

Nebojša Taušan, Jouni Markkula, Pasi Kuvaja et al.

Software companies that develop their products on a basis of service-oriented architecture (SOA) can expect various improvements as a result of choreography. Current choreography practices, however, are not yet used extensively in the embedded systems domain even though SOA is increasingly used in this domain. The objective of this study is to identify current features of the use of choreography in the embedded systems domain for practitioners and researchers by systematically analysing current developments in the scientific literature, strategies for choreography adaption, choreography specification and execution types, and implicit assumptions about choreography. To fulfil this objective, a systematic literature review of scientific publications that focus on the use of choreography in the embedded systems domain was carried out. After screening, 48 publications were selected as primary studies and analysed using thematic synthesis. The main results of the study showed that there are differences in how choreography is used in embedded and non-embedded systems domain. In the embedded systems domain, it is used to capture the service interactions of a single organisation, while, for example, in the enterprise systems domain it captures the service interactions among multiple organisations. Additionally, the results indicate that the use of choreography can lead to improvements in system performance and that the languages that are used for choreography modelling in the embedded systems domain are insufficiently expressive to capture the complexities that are typical in this domain. The study results facilitate the work of practitioners by allowing them to make informed decisions about the applicability of choreography in their organisations.