Stefan Wagner

Umm-e-Habiba, Justus Bogner, Stefan Wagner

With the recent proliferation of artificial intelligence systems, there has been a surge in the demand for explainability of these systems. Explanations help to reduce system opacity, support transparency, and increase stakeholder trust. In this position paper, we discuss synergies between requirements engineering (RE) and Explainable AI (XAI). We highlight challenges in the field of XAI, and propose a framework and research directions on how RE practices can help to mitigate these challenges.

Emad Shihab, Stefan Wagner, Marco A. Gerosa et al.

We are witnessing a massive adoption of software engineering bots, applications that react to events triggered by tools and messages posted by users and run automated tasks in response, in a variety of domains. This thematic issues describes experiences and challenges with these bots.

Umm-e- Habiba, Markus Haug, Justus Bogner et al.

Artificial intelligence (AI) permeates all fields of life, which resulted in new challenges in requirements engineering for artificial intelligence (RE4AI), e.g., the difficulty in specifying and validating requirements for AI or considering new quality requirements due to emerging ethical implications. It is currently unclear if existing RE methods are sufficient or if new ones are needed to address these challenges. Therefore, our goal is to provide a comprehensive overview of RE4AI to researchers and practitioners. What has been achieved so far, i.e., what practices are available, and what research gaps and challenges still need to be addressed? To achieve this, we conducted a systematic mapping study combining query string search and extensive snowballing. The extracted data was aggregated, and results were synthesized using thematic analysis. Our selection process led to the inclusion of 126 primary studies. Existing RE4AI research focuses mainly on requirements analysis and elicitation, with most practices applied in these areas. Furthermore, we identified requirements specification, explainability, and the gap between machine learning engineers and end-users as the most prevalent challenges, along with a few others. Additionally, we proposed seven potential research directions to address these challenges. Practitioners can use our results to identify and select suitable RE methods for working on their AI-based systems, while researchers can build on the identified gaps and research directions to push the field forward.

Ankita Paul, Stefan Wagner, Anup Das

Feedback-driven recurrent spiking neural networks (RSNNs) are powerful computational models that can mimic dynamical systems. However, the presence of a feedback loop from the readout to the recurrent layer de-stabilizes the learning mechanism and prevents it from converging. Here, we propose a supervised training procedure for RSNNs, where a second network is introduced only during the training, to provide hint for the target dynamics. The proposed training procedure consists of generating targets for both recurrent and readout layers (i.e., for a full RSNN system). It uses the recursive least square-based First-Order and Reduced Control Error (FORCE) algorithm to fit the activity of each layer to its target. The proposed full-FORCE training procedure reduces the amount of modifications needed to keep the error between the output and target close to zero. These modifications control the feedback loop, which causes the training to converge. We demonstrate the improved performance and noise robustness of the proposed full-FORCE training procedure to model 8 dynamical systems using RSNNs with leaky integrate and fire (LIF) neurons and rate coding. For energy-efficient hardware implementation, an alternative time-to-first-spike (TTFS) coding is implemented for the full- FORCE training procedure. Compared to rate coding, full-FORCE with TTFS coding generates fewer spikes and facilitates faster convergence to the target dynamics.

Süeda Özkaya, Santiago Berrezueta-Guzman, Stefan Wagner

The integration of Large Language Models (LLMs) into Virtual Reality (VR) games marks a paradigm shift in the design of immersive, adaptive, and intelligent digital experiences. This paper presents a comprehensive review of recent research at the intersection of LLMs and VR, examining how these models are transforming narrative generation, non-player character (NPC) interactions, accessibility, personalization, and game mastering. Drawing from an analysis of 62 peer reviewed studies published between 2018 and 2025, we identify key application domains ranging from emotionally intelligent NPCs and procedurally generated storytelling to AI-driven adaptive systems and inclusive gameplay interfaces. We also address the major challenges facing this convergence, including real-time performance constraints, memory limitations, ethical risks, and scalability barriers. Our findings highlight that while LLMs significantly enhance realism, creativity, and user engagement in VR environments, their effective deployment requires robust design strategies that integrate multimodal interaction, hybrid AI architectures, and ethical safeguards. The paper concludes by outlining future research directions in multimodal AI, affective computing, reinforcement learning, and open-source development, aiming to guide the responsible advancement of intelligent and inclusive VR systems.

Rainer Niedermayr, Stefan Wagner

Mutation testing is a means to assess the effectiveness of a test suite and its outcome is considered more meaningful than code coverage metrics. However, despite several optimizations, mutation testing requires a significant computational effort and has not been widely adopted in industry. Therefore, we study in this paper whether test effectiveness can be approximated using a more light-weight approach. We hypothesize that a test case is more likely to detect faults in methods that are close to the test case on the call stack than in methods that the test case accesses indirectly through many other methods. Based on this hypothesis, we propose the minimal stack distance between test case and method as a new test measure, which expresses how close any test case comes to a given method, and study its correlation with test effectiveness. We conducted an empirical study with 21 open-source projects, which comprise in total 1.8 million LOC, and show that a correlation exists between stack distance and test effectiveness. The correlation reaches a strength up to 0.58. We further show that a classifier using the minimal stack distance along with additional easily computable measures can predict the mutation testing result of a method with 92.9% precision and 93.4% recall. Hence, such a classifier can be taken into consideration as a light-weight alternative to mutation testing or as a preceding, less costly step to that.

Rainer Niedermayr, Tobias Röhm, Stefan Wagner

Background. Test resources are usually limited and therefore it is often not possible to completely test an application before a release. To cope with the problem of scarce resources, development teams can apply defect prediction to identify fault-prone code regions. However, defect prediction tends to low precision in cross-project prediction scenarios. Aims. We take an inverse view on defect prediction and aim to identify methods that can be deferred when testing because they contain hardly any faults due to their code being "trivial". We expect that characteristics of such methods might be project-independent, so that our approach could improve cross-project predictions. Method. We compute code metrics and apply association rule mining to create rules for identifying methods with low fault risk. We conduct an empirical study to assess our approach with six Java open-source projects containing precise fault data at the method level. Results. Our results show that inverse defect prediction can identify approx. 32-44% of the methods of a project to have a low fault risk; on average, they are about six times less likely to contain a fault than other methods. In cross-project predictions with larger, more diversified training sets, identified methods are even eleven times less likely to contain a fault. Conclusions. Inverse defect prediction supports the efficient allocation of test resources by identifying methods that can be treated with less priority in testing activities and is well applicable in cross-project prediction scenarios.

Kai Mindermann, Stefan Wagner

Context: Cryptographic APIs are said to be not usable and researchers suggest to add example code to the documentation. Aim: We wanted to create a free platform for cryptographic code examples that improves the usability and security of created applications by non security experts. Method: We created the open-source web platform CryptoExamples and conducted a controlled experiment where 58 students added symmetric encryption to a Java program. We then measured the usability and security. Results: The participants who used the platform were not only significantly more effective (+73 %) but also their code contained significantly less possible security vulnerabilities (-66 %). Conclusions: With CryptoExamples the gap between hard to change API documentation and the need for complete and secure code examples can be closed. Still, the platform needs more code examples.

Rainer Niedermayr, Tobias Röhm, Stefan Wagner

Test resources are usually limited and therefore it is often not possible to completely test an application before a release. Therefore, testers need to focus their activities on the relevant code regions. In this paper, we introduce an inverse defect prediction approach to identify methods that contain hardly any faults. We applied our approach to six Java open-source projects and show that on average 31.6% of the methods of a project have a low fault risk; they contain in total, on average, only 5.8% of all faults. Furthermore, the results suggest that, unlike defect prediction, our approach can also be applied in cross-project prediction scenarios. Therefore, inverse defect prediction can help prioritize untested code areas and guide testers to increase the fault detection probability.

Verena Käfer, Daniel Graziotin, Ivan Bogicevic et al.

Communication is essential in software engineering. Especially in distributed open-source teams, communication needs to be supported by channels including mailing lists, forums, issue trackers, and chat systems. Yet, we do not have a clear understanding of which communication channels stakeholders in open-source projects use. In this study, we fill the knowledge gap by investigating a statistically representative sample of 400 GitHub projects. We discover the used communication channels by regular expressions on project data. We show that (1) half of the GitHub projects use observable communication channels; (2) GitHub Issues, e-mail addresses, and the modern chat system Gitter are the most common channels; (3) mailing lists are only in place five and have a lower market share than all modern chat systems combined.

Klaus Lochmann, Jasmin Ramadani, Stefan Wagner

The concept of software quality is very complex and has many facets. Reflecting all these facets and at the same time measuring everything related to these facets results in comprehensive but large quality models and extensive measurements. In contrast, there are also many smaller, focused quality models claiming to evaluate quality with few measures. We investigate if and to what extent it is possible to build a focused quality model with similar evaluation results as a comprehensive quality model but with far less measures needed to be collected and, hence, reduced effort. We make quality evaluations with the comprehensive Quamoco base quality model and build focused quality models based on the same set of measures and data from over 2,000 open source systems. We analyse the ability of the focused model to predict the results of the Quamoco model by comparing them with a random predictor as a baseline. We calculate the standardised accuracy measure SA and effect sizes. We found that for the Quamoco model and its 378 automatically collected measures, we can build a focused model with only 10 measures but an accuracy of 61% and a medium to high effect size. We conclude that we can build focused quality models to get an impression of a system's quality similar to comprehensive models. However, when including manually collected measures, the accuracy of the models stayed below 50%. Hence, manual measures seem to have a high impact and should therefore not be ignored in a focused model.

Asim Abdulkhaleq, Stefan Wagner

Software safety is a crucial aspect during the development of modern safety-critical systems. Software is becoming responsible for most of the critical functions of systems. Therefore, the software components in the systems need to be tested extensively against their safety requirements to ensure a high level of system safety. However, performing testing exhaustively to test all software behaviours is impossible. Numerous testing approaches exist. However, they do not directly concern the information derived during the safety analysis. STPA (Systems-Theoretic Process Analysis) is a unique safety analysis approach based on system and control theory, and was developed to identify unsafe scenarios of a complex system including software. In this paper, we present a systematic and semi-automatic testing approach based on STPA to generate test cases from the STPA safety analysis results to help software and safety engineers to recognize and reduce the associated software risks. We also provide an open-source safety-based testing tool called STPA TCGenerator to support the proposed approach. We illustrate the proposed approach with a prototype of a software of the Adaptive Cruise Control System (ACC) with a stop-and-go function with a Lego-Mindstorms EV3 robot.

Stefan Wagner

Context: Software quality is a complex concept. Therefore, assessing and predicting it is still challenging in practice as well as in research. Activity-based quality models break down this complex concept into concrete definitions, more precisely facts about the system, process, and environment as well as their impact on activities performed on and with the system. However, these models lack an operationalisation that would allow them to be used in assessment and prediction of quality. Bayesian networks have been shown to be a viable means for this task incorporating variables with uncertainty. Objective: The qualitative knowledge contained in activity-based quality models are an abundant basis for building Bayesian networks for quality assessment. This paper describes a four-step approach for deriving systematically a Bayesian network from an assessment goal and a quality model. Method: The four steps of the approach are explained in detail and with running examples. Furthermore, an initial evaluation is performed, in which data from NASA projects and an open source system is obtained. The approach is applied to this data and its applicability is analysed. Results: The approach is applicable to the data from the NASA projects and the open source system. However, the predictive results vary depending on the availability and quality of the data, especially the underlying general distributions. Conclusion: The approach is viable in a realistic context but needs further investigation in case studies in order to analyse its predictive validity.

Stefan Wagner, Andreas Goeb, Lars Heinemann et al.

Software quality models provide either abstract quality characteristics or concrete quality measurements; there is no seamless integration of these two aspects. Reasons for this include the complexity of quality and the various quality profiles in different domains which make it difficult to build operationalised quality models. In the project Quamoco, we developed a comprehensive approach for closing this gap. It combined constructive research, which involved quality experts from academia and industry in workshops, sprint work and reviews, with empirical studies. All deliverables within the project were peer-reviewed by two project members from a different area. Most deliverables were developed in two or three iterations and underwent an evaluation. We contribute a comprehensive quality modelling and assessment approach: (1) A meta quality model defines the structure of operationalised quality models. It includes the concept of a product factor, which bridges the gap between concrete measurements and abstract quality aspects, and allows modularisation to create modules for specific domains. (2) A largely technology-independent base quality model reduces the effort and complexity of building quality models for specific domains. For Java and C# systems, we refined it with about 300 concrete product factors and 500 measures. (3) A concrete and comprehensive quality assessment approach makes use of the concepts in the meta-model. (4) An empirical evaluation of the above results using real-world software systems. (5) The extensive, open-source tool support is in a mature state. (6) The model for embedded software systems is a proof-of-concept for domain-specific quality models. We provide a broad basis for the development and application of quality models in industrial practice as well as a basis for further extension, validation and comparison with other approaches in research.

Florian Deissenboeck, Stefan Wagner, Markus Pizka et al.

Over time, software systems suffer gradual quality decay and therefore costs can rise if organizations fail to take proactive countermeasures. Quality control is the first step to avoiding this cost trap. Continuous quality assessments help users identify quality problems early, when their removal is still inexpensive; they also aid decision making by providing an integrated view of a software system's current status. As a side effect, continuous and timely feedback helps developers and maintenance personnel improve their skills and thereby decreases the likelihood of future quality defects. To make regular quality control feasible, it must be highly automated, and assessment results must be presented in an aggregated manner to avoid overwhelming users with data. This article offers an overview of tools that aim to address these issues. The authors also discuss their own flexible, open-source toolkit, which supports the creation of dashboards for quality control.

Stefan Wagner

Software development needs continuous quality control for a timely detection and removal of quality problems. This includes frequent quality assessments, which need to be automated as far as possible to be feasible. One way of automation in assessing the security of software are application scanners that test an executing software for vulnerabilities. At present, common quality assessments do not integrate such scanners for giving an overall quality statement. This paper presents an integration of application scanners into a general quality assessment method based on explicit quality models and Bayesian nets. Its applicability and the detection capabilities of common scanners are investigated in a case study with two open-source web shops.

Marcos Kalinowski, Daniel Mendez, Görkem Giray et al.

Context: Machine learning (ML)-enabled systems are being increasingly adopted by companies aiming to enhance their products and operational processes. Objective: This paper aims to deliver a comprehensive overview of the current status quo of engineering ML-enabled systems and lay the foundation to steer practically relevant and problem-driven academic research. Method: We conducted an international survey to collect insights from practitioners on the current practices and problems in engineering ML-enabled systems. We received 188 complete responses from 25 countries. We conducted quantitative statistical analyses on contemporary practices using bootstrapping with confidence intervals and qualitative analyses on the reported problems using open and axial coding procedures. Results: Our survey results reinforce and extend existing empirical evidence on engineering ML-enabled systems, providing additional insights into typical ML-enabled systems project contexts, the perceived relevance and complexity of ML life cycle phases, and current practices related to problem understanding, model deployment, and model monitoring. Furthermore, the qualitative analysis provides a detailed map of the problems practitioners face within each ML life cycle phase and the problems causing overall project failure. Conclusions: The results contribute to a better understanding of the status quo and problems in practical environments. We advocate for the further adaptation and dissemination of software engineering practices to enhance the engineering of ML-enabled systems.

Tobias Eisenreich, Nicholas Friedlaender, Stefan Wagner

Use case modeling employs user-centered scenarios to outline system requirements. These help to achieve consensus among relevant stakeholders. Because the manual creation of use case models is demanding and time-consuming, it is often skipped in practice. This study explores the potential of Large Language Models (LLMs) to assist in this tedious process. The proposed method integrates an open-weight LLM to systematically extract actors and use cases from software requirements with advanced prompt engineering techniques. The method is evaluated using an exploratory study conducted with five professional software engineers, which compares traditional manual modeling to the proposed LLM-based approach. The results show a substantial acceleration, reducing the modeling time by 60\%. At the same time, the model quality remains on par. Besides improving the modeling efficiency, the participants indicated that the method provided valuable guidance in the process.

Santiago Berrezueta-Guzman, Stephan Krusche, Stefan Wagner

The rapid adoption of AI powered coding assistants like ChatGPT and other coding copilots is transforming programming education, raising questions about assessment practices, academic integrity, and skill development. As educators seek alternatives to traditional grading methods susceptible to AI enabled plagiarism, structured peer assessment could be a promising strategy. This paper presents an empirical study of a rubric based, anonymized peer review process implemented in a large introductory programming course. Students evaluated each other's final projects (2D game), and their assessments were compared to instructor grades using correlation, mean absolute error, and root mean square error (RMSE). Additionally, reflective surveys from 47 teams captured student perceptions of fairness, grading behavior, and preferences regarding grade aggregation. Results show that peer review can approximate instructor evaluation with moderate accuracy and foster student engagement, evaluative thinking, and interest in providing good feedback to their peers. We discuss these findings for designing scalable, trustworthy peer assessment systems to face the age of AI assisted coding.

Denis Schwachhofer, Peter Domanski, Steffen Becker et al.

System-Level Test (SLT) has been a part of the test flow for integrated circuits for over a decade and still gains importance. However, no systematic approaches exist for test program generation, especially targeting non-functional properties of the Device under Test (DUT). Currently, test engineers manually compose test suites from off-the-shelf software, approximating the end-user environment of the DUT. This is a challenging and tedious task that does not guarantee sufficient control over non-functional properties. This paper proposes Large Language Models (LLMs) to generate test programs. We take a first glance at how pre-trained LLMs perform in test program generation to optimize non-functional properties of the DUT. Therefore, we write a prompt to generate C code snippets that maximize the instructions per cycle of a super-scalar, out-of-order architecture in simulation. Additionally, we apply prompt and hyperparameter optimization to achieve the best possible results without further training.

Esra Mehmedova, Santiago Berrezueta-Guzman, Stefan Wagner

The Apple Vision Pro is equipped with accurate eye-tracking capabilities, yet the privacy restrictions on the device prevent direct access to continuous user gaze data. This study introduces iTrace, a novel application that overcomes these limitations through click-based gaze extraction techniques, including manual methods like a pinch gesture, and automatic approaches utilizing dwell control or a gaming controller. We developed a system with a client-server architecture that captures the gaze coordinates and transforms them into dynamic heatmaps for video and spatial eye tracking. The system can generate individual and averaged heatmaps, enabling analysis of personal and collective attention patterns. To demonstrate its effectiveness and evaluate the usability and performance, a study was conducted with two groups of 10 participants, each testing different clicking methods. The 8BitDo controller achieved higher average data collection rates at 14.22 clicks/s compared to 0.45 clicks/s with dwell control, enabling significantly denser heatmap visualizations. The resulting heatmaps reveal distinct attention patterns, including concentrated focus in lecture videos and broader scanning during problem-solving tasks. By allowing dynamic attention visualization while maintaining a high gaze precision of 91 %, iTrace demonstrates strong potential for a wide range of applications in educational content engagement, environmental design evaluation, marketing analysis, and clinical cognitive assessment. Despite the current gaze data restrictions on the Apple Vision Pro, we encourage developers to use iTrace only in research settings.

Tobias Eisenreich, Sandro Speth, Stefan Wagner

Designing domain models and software architectures represents a significant challenge in software development, as the resulting architectures play a vital role in fulfilling the system's quality of service. Due to time pressure, architects often model only one architecture based on their known limited domain understanding, patterns, and experience instead of thoroughly analyzing the domain and evaluating multiple candidates, selecting the best fitting. Existing approaches try to generate domain models based on requirements, but still require time-consuming manual effort to achieve good results. Therefore, in this vision paper, we propose a method to generate software architecture candidates semi-automatically based on requirements using artificial intelligence techniques. We further envision an automatic evaluation and trade-off analysis of the generated architecture candidates using, e.g., the architecture trade-off analysis method combined with large language models and quantitative analyses. To evaluate this approach, we aim to analyze the quality of the generated architecture models and the efficiency and effectiveness of our proposed process by conducting qualitative studies.

Stefan Wagner, Marvin Wyrich

Literature and intuition suggest that a developer's intelligence and personality have an impact on their performance in comprehending source code. Researchers made this suggestion in the past when discussing threats to validity of their study results. However, the lack of studies investigating the relationship of intelligence and personality to performance in code comprehension makes scientifically sound reasoning about their influence difficult. We conduct the first empirical evaluation, a correlational study with undergraduates, to investigate the correlation of intelligence and personality with performance in code comprehension, that is with correctness in answering comprehension questions on code snippets. We found that personality traits are unlikely to impact code comprehension performance, at least not considered in isolation. Conscientiousness, in combination with other factors, however, explains some of the variance in code comprehension performance. For intelligence, significant small to moderate positive effects on code comprehension performance were found for three of four factors measured, i.e., fluid intelligence, visual perception, and cognitive speed. Crystallized intelligence has a positive but statistically insignificant effect on code comprehension performance. According to our results, several intelligence facets as well as the personality trait conscientiousness are potential confounders that should not be neglected in code comprehension studies of individual performance and should be controlled for via an appropriate study design. We call for the conduct of further studies on the relationship between intelligence and personality with code comprehension, in part because code comprehension involves more facets than we can measure in a single study and because our regression model explains only a small portion of the variance in code comprehension performance.

Mohammad Kasra Habib, Stefan Wagner, Daniel Graziotin

Requirements Engineering (RE) is the initial step towards building a software system. The success or failure of a software project is firmly tied to this phase, based on communication among stakeholders using natural language. The problem with natural language is that it can easily lead to different understandings if it is not expressed precisely by the stakeholders involved, which results in building a product different from the expected one. Previous work proposed to enhance the quality of the software requirements detecting language errors based on ISO 29148 requirements language criteria. The existing solutions apply classical Natural Language Processing (NLP) to detect them. NLP has some limitations, such as domain dependability which results in poor generalization capability. Therefore, this work aims to improve the previous work by creating a manually labeled dataset and using ensemble learning, Deep Learning (DL), and techniques such as word embeddings and transfer learning to overcome the generalization problem that is tied with classical NLP and improve precision and recall metrics using a manually labeled dataset. The current findings show that the dataset is unbalanced and which class examples should be added more. It is tempting to train algorithms even if the dataset is not considerably representative. Whence, the results show that models are overfitting; in Machine Learning this issue is solved by adding more instances to the dataset, improving label quality, removing noise, and reducing the learning algorithms complexity, which is planned for this research.

Silverio Martínez-Fernández, Justus Bogner, Xavier Franch et al.

AI-based systems are software systems with functionalities enabled by at least one AI component (e.g., for image- and speech-recognition, and autonomous driving). AI-based systems are becoming pervasive in society due to advances in AI. However, there is limited synthesized knowledge on Software Engineering (SE) approaches for building, operating, and maintaining AI-based systems. To collect and analyze state-of-the-art knowledge about SE for AI-based systems, we conducted a systematic mapping study. We considered 248 studies published between January 2010 and March 2020. SE for AI-based systems is an emerging research area, where more than 2/3 of the studies have been published since 2018. The most studied properties of AI-based systems are dependability and safety. We identified multiple SE approaches for AI-based systems, which we classified according to the SWEBOK areas. Studies related to software testing and software quality are very prevalent, while areas like software maintenance seem neglected. Data-related issues are the most recurrent challenges. Our results are valuable for: researchers, to quickly understand the state of the art and learn which topics need more research; practitioners, to learn about the approaches and challenges that SE entails for AI-based systems; and, educators, to bridge the gap among SE and AI in their curricula.

Jinghua Yu, Stefan Wagner, Bowen Wang et al.

The innovations of vehicle connectivity have been increasing dramatically to enhance the safety and user experience of driving, while the rising numbers of interfaces to the external world also bring security threats to vehicles. Many security countermeasures have been proposed and discussed to protect the systems and services against attacks. To provide an overview of the current states in this research field, we conducted a systematic mapping study on the topic area "security countermeasures of in-vehicle communication systems". 279 papers are identified based on the defined study identification strategy and criteria. We discussed four research questions related to the security countermeasures, validation methods, publication patterns, and research trends and gaps based on the extracted and classified data. Finally, we evaluated the validity threats, the study identification results, and the whole mapping process. We found that the studies in this topic area are increasing rapidly in recent years. However, there are still gaps in various subtopics like automotive Ethernet security, anomaly reaction, and so on. This study reviews the target field not only related to research findings but also research activities, which can help identify research gaps at a high level and inspire new ideas for future work.

Ilia Polian, Jens Anders, Steffen Becker et al.

System-level test, or SLT, is an increasingly important process step in today's integrated circuit testing flows. Broadly speaking, SLT aims at executing functional workloads in operational modes. In this paper, we consolidate available knowledge about what SLT is precisely and why it is used despite its considerable costs and complexities. We discuss the types or failures covered by SLT, and outline approaches to quality assessment, test generation and root-cause diagnosis in the context of SLT. Observing that the theoretical understanding for all these questions has not yet reached the level of maturity of the more conventional structural and functional test methods, we outline new and promising directions for methodical developments leveraging on recent findings from software engineering.

Ana Cristina Franco da Silva, Stefan Wagner, Eddie Lazebnik et al.

A Digital Twin (DT) is a digital representation of a physical object used to simulate it before it is built or to predict failures after the object is deployed. In this article, we introduce our approach, which applies the concept of a Cyber Digital Twin (CDT) to automotive software for the purpose of security analysis. In our approach, automotive firmware is transformed into a CDT, which contains automatically extracted, security-relevant information from the firmware. Based on the CDT, we evaluate security requirements through automated analysis and requirements verification using policy enforcement checks and vulnerabilities detection. The evaluation of a CDT is conducted continuously integrating new checks derived from new security requirements and from newly disclosed vulnerabilities. We applied our approach to about 100 automotive firmwares. In average, about 600 publicly disclosed vulnerabilities and 80 unknown weaknesses were detected per firmware in the pre-production phase. Therefore, the use of a CDT enables efficient continuous verification of security requirements.

Jonas Fritzsch, Marvin Wyrich, Justus Bogner et al.

Technologies play an important role in the hiring process for software professionals. Within this process, several studies revealed misconceptions and bad practices which lead to suboptimal recruitment experiences. In the same context, grey literature anecdotally coined the term Résumé-Driven Development (RDD), a phenomenon describing the overemphasis of trending technologies in both job offerings and resumes as an interaction between employers and applicants. While RDD has been sporadically mentioned in books and online discussions, there are so far no scientific studies on the topic, despite its potential negative consequences. We therefore empirically investigated this phenomenon by surveying 591 software professionals in both hiring (130) and technical (558) roles and identified RDD facets in substantial parts of our sample: 60% of our hiring professionals agreed that trends influence their job offerings, while 82% of our software professionals believed that using trending technologies in their daily work makes them more attractive for prospective employers. Grounded in the survey results, we conceptualize a theory to frame and explain Résumé-Driven Development. Finally, we discuss influencing factors and consequences and propose a definition of the term. Our contribution provides a foundation for future research and raises awareness for a potentially systemic trend that may broadly affect the software industry.

Anneke Pehmöller, Frank Salger, Stefan Wagner

Although testing is critical in GSD, its application in this context has not been deeply investigated so far. This work investigates testing in GSD. It provides support for test managers acting in a globally distributed environment. With this it closes a gap. The leading question is "What problems exist in testing in GSD and how can they be addressed in projects?" Decomposing this question we a) identify problems of testing in GSD projects and b) provide good practices to support practitioners in testing in GSD projects. The research is realized in the context of Capgemini Germany. Our contribution to solving the stated research problem is a collection of 16 patterns for testing in GSD projects. For practitioners the usage of the patterns is simplified by various views on the patterns. Herewith we stipulate research and support project managers and test managers in the realization of testing in GSD projects.

Tobias Schmid, Stefanie Schraufstetter, Jonas Fritzsch et al.

A fail-operational system for highly automated driving must complete the driving task even in the presence of a failure. This requires redundant architectures and a mechanism to reconfigure the system in case of a failure. Therefore, an arbitration logic is used. For functional safety, the switch-over to a fall-back level must be conducted in the presence of any electric and electronic failure. To provide evidence for a safety argumentation in compliance with ISO 26262, verification of the arbitration logic is necessary. The verification process provides confirmation of the correct failure reactions and that no unintended system states are attainable. Conventional safety analyses, such as the failure mode and effect analysis, have its limits in this regard. We present an analytical approach based on formal verification, in particular model checking, to verify the fail-operational behaviour of a driving system. For that reason, we model the system behaviour and the relevant architecture and formally specify the safety requirements. The scope of the analysis is defined according to the requirements of ISO 26262. We verify a fail-operational arbitration logic for highly automated driving in compliance with the industry standard. Our results show that formal methods for safety evaluation in automotive fail-operational driving systems can be successfully applied. We were able to detect failures, which would have been overlooked by other analyses and thus contribute to the development of safety critical functions.

Marvin Wyrich, Andreas Preikschat, Daniel Graziotin et al.

Static code analysis tools and integrated development environments present developers with quality-related software metrics, some of which describe the understandability of source code. Software metrics influence overarching strategic decisions that impact the future of companies and the prioritization of everyday software development tasks. Several software metrics, however, lack in validation: we just choose to trust that they reflect what they are supposed to measure. Some of them were even shown to not measure the quality aspects they intend to measure. Yet, they influence us through biases in our cognitive-driven actions. In particular, they might anchor us in our decisions. Whether the anchoring effect exists with software metrics has not been studied yet. We conducted a randomized and double-blind experiment to investigate the extent to which a displayed metric value for source code comprehensibility anchors developers in their subjective rating of source code comprehensibility, whether performance is affected by the anchoring effect when working on comprehension tasks, and which individual characteristics might play a role in the anchoring effect. We found that the displayed value of a comprehensibility metric has a significant and large anchoring effect on a developer's code comprehensibility rating. The effect does not seem to affect the time or correctness when working on comprehension questions related to the code snippets under study. Since the anchoring effect is one of the most robust cognitive biases, and we have limited understanding of the consequences of the demonstrated manipulation of developers by non-validated metrics, we call for an increased awareness of the responsibility in code quality reporting and for corresponding tools to be based on scientific evidence.

Kristina Marner, Stefan Wagner, Guenther Ruhe

Context: In regulated domains like automotive, release planning is a complex process. The agreement between traditional product development processes for hardware as well as mechanic systems and agile development approaches for software development is a major challenge. Especially the creation and synchronization of a release plan is challenging. Objective: The aim of this work is to present identified stakeholders of a release plan as an appropriate approach to create transparency in release planning in the automotive domain. Method: Action research to elaborate relevant stakeholders for release planning was conducted at Dr. Ing. h. c. F. Porsche AG. Results: We present a detailed overview of identified stakeholders due to release planning as well as their required content and added value regarding to two pilot projects. The results confirm the fact that almost every stakeholder is involved in a release plan in a certain way. Conclusions: Release planning within a complex project environment and complicated customer constellations is difficult to manage. We discuss how the presented stakeholders could meet with the given conditions in the automotive domain. With this contribution, identified stakeholders of release planning from hardware and software point of view is introduced. It helps to reach transparency and to handle the given complexity.

Jinghua Yu, Stefan Wagner, Feng Luo

The in-vehicle diagnostic and software update system, which supports remote diagnostic and Over-The-Air (OTA) software updates, is a critical attack goal in automobiles. Adversaries can inject malicious software into vehicles or steal sensitive information through communication channels. Therefore, security analysis, which identifies potential security issues, needs to be conducted in system design. However, existing security analyses of in-vehicle systems are threat-oriented, which start with threat identification and assess risks by brainstorming. In this paper, a system-oriented approach is proposed on the basis of the System-Theoretic Process Analysis (STPA). The proposed approach extends the original STPA from the perspective of data flows and is applicable for information-flow-based systems. Besides, we propose a general model for in-vehicle diagnostic and software update systems and use it to establish a security analysis guideline. In comparison with threat-oriented approaches, the proposed approach shifts from focusing on threats to system vulnerabilities and seems to be efficient to prevent the system from known or even unknown threats. Furthermore, as an extension of the STPA, which has been proven to be applicable to high level designs, the proposed approach can be well integrated into high-level analyses and perform co-design in different disciplines within a unified STPA framework.

Jinghua Yu, Stefan Wagner, Feng Luo

Security analysis is an essential activity in security engineering to identify potential system vulnerabilities and achieve security requirements in the early design phases. Due to the increasing complexity of modern systems, traditional approaches, which only consider component failures and simple cause-and-effect linkages, lack the power to identify insecure incidents caused by complex interactions among physical systems, human and social entities. By contrast, a top-down System-Theoretic Process Analysis for Security (STPA-Sec) approach views losses as resulting from interactions, focuses on controlling system vulnerabilities instead of external threats and is applicable for complex socio-technical systems. In this paper, we proposed an extension of STPA-Sec based on data flow structures to overcome STPA-Sec's limitations and achieve security constraints of information-critical systems systematically. We analyzed a Bluetooth digital key system of a vehicle by using both the proposed and the original approach to investigate the relationship and differences between both approaches as well as their applicability and highlights. To conclude, the proposed approach can identify more information-related problems with technical details and be used with other STPA-based approaches to co-design systems in multi-disciplines under the unified STPA process framework.

Daniel Graziotin, Per Lenberg, Robert Feldt et al.

A meaningful and deep understanding of the human aspects of software engineering (SE) requires psychological constructs to be considered. Psychology theory can facilitate the systematic and sound development as well as the adoption of instruments (e.g., psychological tests, questionnaires) to assess these constructs. In particular, to ensure high quality, the psychometric properties of instruments need evaluation. In this paper, we provide an introduction to psychometric theory for the evaluation of measurement instruments for SE researchers. We present guidelines that enable using existing instruments and developing new ones adequately. We conducted a comprehensive review of the psychology literature framed by the Standards for Educational and Psychological Testing. We detail activities used when operationalizing new psychological constructs, such as item pooling, item review, pilot testing, item analysis, factor analysis, statistical property of items, reliability, validity, and fairness in testing and test bias. We provide an openly available example of a psychometric evaluation based on our guideline. We hope to encourage a culture change in SE research towards the adoption of established methods from psychology. To improve the quality of behavioral research in SE, studies focusing on introducing, validating, and then using psychometric instruments need to be more common.

Simone V. Spiegler, Daniel Graziotin, Christoph Heinecke et al.

A number of qualitative studies find that team leadership is one essential success factor for evolving into a mature agile team. One such qualitative study suggests the 9-Factor Theory of Scrum Master roles, which claims that the Scrum Master performs a set of 9 leadership roles which are transferred to the team over time (Spiegler et al., 2019). We aimed at conducting a quantitative exploration that examines the presence and change of the 9-Factory Theory in relation to team maturity. We conducted an online survey with 67 individuals at the conglomerate Robert Bosch GmbH. Descriptive statistics reveal that the Scrum Master and the agile team score differently on the 9 factors and that the Scrum Master role is most often distributed in teams that had been working between 3 and 5 months in an agile manner. Yet, we also find that the leadership roles predominantly remain with one dedicated Scrum Master. Based on our results we suggest to group the 9-Factor Theory into three clusters: the Scrum Master is rather linked to psychological team factors (1), while the team tends to be linked to rather product-related factors (2). Organizational factors (3) are less often present. Our practical implications suggest an extension of the Scrum Master description. Furthermore, our study lays groundwork for future quantitative testing of leadership in agile teams.

Justus Bogner, Jonas Fritzsch, Stefan Wagner et al.

While Microservices promise several beneficial characteristics for sustainable long-term software evolution, little empirical research covers what concrete activities industry applies for the evolvability assurance of Microservices and how technical debt is handled in such systems. Since insights into the current state of practice are very important for researchers, we performed a qualitative interview study to explore applied evolvability assurance processes, the usage of tools, metrics, and patterns, as well as participants' reflections on the topic. In 17 semi-structured interviews, we discussed 14 different Microservice-based systems with software professionals from 10 companies and how the sustainable evolution of these systems was ensured. Interview transcripts were analyzed with a detailed coding system and the constant comparison method. We found that especially systems for external customers relied on central governance for the assurance. Participants saw guidelines like architectural principles as important to ensure a base consistency for evolvability. Interviewees also valued manual activities like code review or boy scouting, even though automation and tool support was described as very important. Source code quality was the primary target for the usage of tools and metrics. Despite most reported issues being related to Architectural Technical Debt (ATD), our participants did not apply any architectural or service-oriented tools and metrics. While participants generally saw their Microservices as evolvable, service cutting and finding an appropriate service granularity with low coupling and high cohesion were reported as challenging. Future Microservices research in the areas of evolution and technical debt should take these findings and industry sentiments into account.

Daniel Méndez Fernández, Xavier Franch, Norbert Seyff et al.

Nowadays, there exist a plethora of different educational syllabi for Requirements Engineering (RE), all aiming at incorporating practically relevant educational units (EUs). Many of these syllabi are based, in one way or the other, on the syllabi provided by the International Requirements Engineering Board (IREB), a non-profit organisation devoted to standardised certification programs for RE. IREB syllabi are developed by RE experts and are, thus, based on the assumption that they address topics of practical relevance. However, little is known about to what extent practitioners actually perceive those contents as useful. We have started a study to investigate the relevance of the EUs included in the IREB Foundation Level certification programme. In a first phase reported in this paper, we have surveyed practitioners mainly from DACH countries (Germany, Austria and Switzerland) participating in the IREB certification. Later phases will widen the scope both by including other countries and by not requiring IREB-certified participants. The results shall foster a critical reflection on the practical relevance of EUs built upon the de-facto standard syllabus of IREB.

Simone V. Spiegler, Christoph Heinecke, Stefan Wagner

Motivation: How immature teams can become agile is a question that puzzles practitioners and researchers alike. Scrum is one method that supports agile working. Empirical research on the Scrum Master role remains scarce and reveals contradicting results. While the Scrum Master role is often centred on one person in rather immature teams, the role is expected to be shared among multiple members in mature teams. Objective: Therefore, we aim to understand how the Scrum Master role changes while the team matures. Method: We applied Grounded Theory and conducted qualitative interviews with 53 practitioners of 29 software and non-software project teams from Robert Bosch GmbH. Results: We discovered that Scrum Masters initially plays nine leadership roles which they transfer to the team while it matures. Roles can be transferred by providing a leadership gap, which allows team members to take on a leadership role, and by providing an internal team environment with communication on equal terms, psychological safety, transparency, shared understanding, shared purpose and self-efficacy. Conclusion: The Scrum Master role changes while the team matures. Trust and freedom to take over a leadership role in teams are essential enablers. Our results support practitioners in implementing agile teams in established companies.

Stefan Wagner, Matthias Tichy, Michael Felderer et al.

Software is the main innovation driver in many different areas, like cloud services, autonomous driving, connected medical devices, and high-frequency trading. All these areas have in common that they require high dependability. In this paper, we discuss challenges and research directions imposed by these new areas on guaranteeing the dependability. On the one hand challenges include characteristics of the systems themselves, e. g., open systems and ad-hoc structures. On the other hand, we see new aspects of dependability like behavioral traceability.

Daniel Kulesz, Stefan Wagner

Spreadsheets (sometimes also called Excel programs) are powerful tools which play a business-critical role in many organizations. However, due to faulty spreadsheets many bad decisions have been taken in recent years. Since then, a number of researchers have been studying spreadsheet errors. However, one issue that hinders discussion among researchers and professionals is the lack of a commonly accepted taxonomy. Albeit a number of taxonomies for spreadsheet errors have been proposed in previous work, a major issue is that they use the term error that itself is already ambiguous. Furthermore, to apply most existing taxonomies, detailed knowledge about the underlying process and knowledge about the "brain state" of the acting spreadsheet users is required. Due to these limitations, known error-like phenomena in freely available spreadsheet corpora cannot be classified with these taxonomies. We propose Asheetoxy, a simple and phenomenon-oriented taxonomy that avoids the problematic term error altogether. An initial study with 7 participants indicates that even non-spreadsheet researchers similarly classify real-world spreadsheet phenomena using Asheetoxy.

Kai Mindermann, Philipp Keck, Stefan Wagner

Context: Poor usability of cryptographic APIs is a severe source of vulnerabilities. Aim: We wanted to find out what kind of cryptographic libraries are present in Rust and how usable they are. Method: We explored Rust's cryptographic libraries through a systematic search, conducted an exploratory study on the major libraries and a controlled experiment on two of these libraries with 28 student participants. Results: Only half of the major libraries explicitly focus on usability and misuse resistance, which is reflected in their current APIs. We found that participants were more successful using rust-crypto which we considered less usable than ring before the experiment. Conclusion: We discuss API design insights and make recommendations for the design of crypto libraries in Rust regarding the detail and structure of the documentation, higher-level APIs as wrappers for the existing low-level libraries, and selected, good-quality example code to improve the emerging cryptographic libraries of Rust.

Peter Braun, Jan Philipps, Bernhard Schätz et al.

Safety cases become increasingly important for software certification. Models play a crucial role in building and combining information for the safety case. This position paper sketches an ideal model-based safety case with defect hypotheses and failure characterisations. From this, open research issues are derived.

Stefan Wagner, Daniel Méndez Fernández, Michael Felderer et al.

Requirements Engineering (RE) has established itself as a software engineering discipline during the past decades. While researchers have been investigating the RE discipline with a plethora of empirical studies, attempts to systematically derive an empirically-based theory in context of the RE discipline have just recently been started. However, such a theory is needed if we are to define and motivate guidance in performing high quality RE research and practice. We aim at providing an empirical and valid foundation for a theory of RE, which helps software engineers establish effective and efficient RE processes. We designed a survey instrument and theory that has now been replicated in 10 countries world-wide. We evaluate the propositions of the theory with bootstrapped confidence intervals and derive potential explanations for the propositions. We report on the underlying theory and the full results obtained from the replication studies with participants from 228 organisations. Our results represent a substantial step forward towards developing an empirically-based theory of RE giving insights into current practices with RE processes. The results reveal, for example, that there are no strong differences between organisations in different countries and regions, that interviews, facilitated meetings and prototyping are the most used elicitation techniques, that requirements are often documented textually, that traces between requirements and code or design documents is common, requirements specifications themselves are rarely changed and that requirements engineering (process) improvement endeavours are mostly intrinsically motivated. Our study establishes a theory that can be used as starting point for many further studies for more detailed investigations. Practitioners can use the results as theory-supported guidance on selecting suitable RE methods and techniques.

Yang Wang, Stefan Wagner

Context: Agile development is in widespread use, even in safety-critical domains. Motivation: However, there is a lack of an ap- propriate safety analysis and verification method in agile development. Objective: In this paper, we investigate the use of Behavior Driven De- velopment (BDD) instead of standard User Acceptance Testing (UAT) for safety verification with System-Theoretic Process Analysis (STPA) for safety analysis in agile development. Method: We evaluate the effect of this combination in a controlled experiment with 44 students in terms of productivity, test thoroughness, fault detection effectiveness and com- munication effectiveness. Results: The results show that BDD is more effective for safety verification regarding the impact on communication effectiveness than standard UAT, whereas productivity, test thorough- ness and fault detection effectiveness show no statistically significant difference in our controlled experiment. Conclusion: The combination of BDD and STPA seems promising with an enhancement on communica- tion, but its impact needs more research.

Stefan Wagner, Melanie Ruhe

Analysing and improving productivity has been one of the main goals of software engineering research since its beginnings. A plethora of studies has been conducted on various factors that resulted in several models for analysis and prediction of productivity. However, productivity is still an issue in current software development and not all factors and their relationships are known. This paper reviews the large body of available literature in order to distill a list of the main factors influencing productivity investigated so far. The measure for importance here is the number of articles a factor is mentioned in. Special consideration is given to soft or human-related factors in software engineering that are often not analysed with equal detail as more technical factors. The resulting list can be used to guide further analysis and as basis for building productivity models.

Elmar Juergens, Florian Deissenboeck, Martin Feilkas et al.

Due to their pivotal role in software engineering, considerable effort is spent on the quality assurance of software requirements specifications. As they are mainly described in natural language, relatively few means of automated quality assessment exist. However, we found that clone detection, a technique widely applied to source code, is promising to assess one important quality aspect in an automated way, namely redundancy that stems from copy&paste operations. This paper describes a large-scale case study that applied clone detection to 28 requirements specifications with a total of 8,667 pages. We report on the amount of redundancy found in real-world specifications, discuss its nature as well as its consequences and evaluate in how far existing code clone detection approaches can be applied to assess the quality of requirements specifications in practice.

Stefan Wagner, Florian Deissenboeck

Abstraction is one of the fundamental concepts of software design. Consequently, the determination of an appropriate abstraction level for the multitude of artefacts that form a software system is an integral part of software engineering. However, the very nature of abstraction in software design and particularly its interrelation with equally important concepts like complexity, specificity or genericity are not fully understood today. As a step towards a better understanding of the trade-offs involved, this paper proposes a distinction of abstraction into two types that have different effects on the specificity and the complexity of artefacts. We discuss the roles of the two types of abstraction in software design and explain the interrelations between abstractness, specificity, and complexity. Furthermore, we illustrate the benefit of the proposed distinction with multiple examples and describe consequences of our findings for software design activities.

Verena Käfer, Daniel Kulesz, Stefan Wagner

The better developers can learn software tools, the faster they can start using them and the more efficiently they can later work with them. Tutorials are supposed to help here. While in the early days of computing, mostly text tutorials were available, nowadays software developers can choose among a huge number of tutorials for almost any popular software tool. However, only little research was conducted to understand how text tutorials differ from other tutorials, which tutorial types are preferred and, especially, which tutorial types yield the best learning experience in terms of efficiency and effectiveness, especially for programmers. To evaluate these questions, we converted an existing video tutorial for a novel software tool into a content-equivalent text tutorial. We then conducted an experiment in three groups where 42 undergraduate students from a software engineering course were commissioned to operate the software tool after using a tutorial: the first group was provided only with the video tutorial, the second group only with the text tutorial and the third group with both. In this context, the differences in terms of efficiency were almost negligible: We could observe that participants using only the text tutorial completed the tutorial faster than the participants with the video tutorial. However, the participants using only the video tutorial applied the learned content faster, achieving roughly the same bottom line performance. We also found that if both tutorial types are offered, participants prefer video tutorials for learning new content but text tutorials for looking up "missed" information. We mainly gathered our data through questionnaires and screen recordings and analyzed it with suitable statistical hypotheses tests. The data is available at [12]. Since producing tutorials requires effort, knowing with which type of tutorial learnability can be increased to which extent has an immense practical relevance. We conclude that in contexts similar to ours, while it would be ideal if software tool makers would offer both tutorial types, it seems more efficient to produce only text tutorials instead of a passive video tutorial - provided you manage to motivate your learners to use them.