Daniel Méndez Fernández

Daniel Méndez Fernández, Daniel Graziotin, Stefan Wagner et al.

Open science describes the movement of making any research artefact available to the public and includes, but is not limited to, open access, open data, and open source. While open science is becoming generally accepted as a norm in other scientific disciplines, in software engineering, we are still struggling in adapting open science to the particularities of our discipline, rendering progress in our scientific community cumbersome. In this chapter, we reflect upon the essentials in open science for software engineering including what open science is, why we should engage in it, and how we should do it. We particularly draw from our experiences made as conference chairs implementing open science initiatives and as researchers actively engaging in open science to critically discuss challenges and pitfalls, and to address more advanced topics such as how and under which conditions to share preprints, what infrastructure and licence model to cover, or how do it within the limitations of different reviewing models, such as double-blind reviewing. Our hope is to help establishing a common ground and to contribute to make open science a norm also in software engineering.

Fabiola Moyón, Daniel Méndez Fernández, Kristian Beckers et al.

Integrating security into agile software development is an open issue for research and practice. Especially in strongly regulated industries, complexity increases not only when scaling agile practices but also when aiming for compliance with security standards. To achieve security compliance in a large-scale agile context, we developed S2C-SAFe: An extension of the Scaled Agile Framework that is compliant to the security standard IEC~62443-4-1 for secure product development. In this paper, we present the framework and its evaluation by agile and security experts within Siemens' large-scale project ecosystem. We discuss benefits and limitations as well as challenges from a practitioners' perspective. Our results indicate that \ssafe contributes to successfully integrating security compliance with lean and agile development in regulated environments. We also hope to raise awareness for the importance and challenges of integrating security in the scope of Continuous Software Engineering.

Stefan Wagner, Daniel Méndez Fernández, Michael Felderer et al.

While being an important and often used research method, survey research has been less often discussed on a methodological level in empirical software engineering than other types of research. This chapter compiles a set of important and challenging issues in survey research based on experiences with several large-scale international surveys. The chapter covers theory building, sampling, invitation and follow-up, statistical as well as qualitative analysis of survey data and the usage of psychometrics in software engineering surveys.

Daniel Méndez Fernández, Xavier Franch, Norbert Seyff et al.

Nowadays, there exist a plethora of different educational syllabi for Requirements Engineering (RE), all aiming at incorporating practically relevant educational units (EUs). Many of these syllabi are based, in one way or the other, on the syllabi provided by the International Requirements Engineering Board (IREB), a non-profit organisation devoted to standardised certification programs for RE. IREB syllabi are developed by RE experts and are, thus, based on the assumption that they address topics of practical relevance. However, little is known about to what extent practitioners actually perceive those contents as useful. We have started a study to investigate the relevance of the EUs included in the IREB Foundation Level certification programme. In a first phase reported in this paper, we have surveyed practitioners mainly from DACH countries (Germany, Austria and Switzerland) participating in the IREB certification. Later phases will widen the scope both by including other countries and by not requiring IREB-certified participants. The results shall foster a critical reflection on the practical relevance of EUs built upon the de-facto standard syllabus of IREB.

Daniel Méndez Fernández, Jan-Hendrik Passoth

Empirical software engineering has received much attention in recent years and coined the shift from a more design-science-driven engineering discipline to an insight-oriented, and theory-centric one. Yet, we still face many challenges, among which some increase the need for interdisciplinary research. This is especially true for the investigation of human-centric aspects of software engineering. Although we can already observe an increased recognition of the need for more interdisciplinary research in (empirical) software engineering, such research configurations come with challenges barely discussed from a scientific point of view. In this position paper, we critically reflect upon the epistemological setting of empirical software engineering and elaborate its configuration as an Interdiscipline. In particular, we (1) elaborate a pragmatic view on empirical research for software engineering reflecting a cyclic process for knowledge creation, (2) motivate a path towards symmetrical interdisciplinary research, and (3) adopt five rules of thumb from other interdisciplinary collaborations in our field before concluding with new emerging challenges. This shall support stopping to treating empirical software engineering as a developing discipline moving towards a paradigmatic stage of normal science, but as a configuration of symmetric interdisciplinary teams and research methods.

Stefan Wagner, Daniel Méndez Fernández, Michael Felderer et al.

Requirements Engineering (RE) has established itself as a software engineering discipline during the past decades. While researchers have been investigating the RE discipline with a plethora of empirical studies, attempts to systematically derive an empirically-based theory in context of the RE discipline have just recently been started. However, such a theory is needed if we are to define and motivate guidance in performing high quality RE research and practice. We aim at providing an empirical and valid foundation for a theory of RE, which helps software engineers establish effective and efficient RE processes. We designed a survey instrument and theory that has now been replicated in 10 countries world-wide. We evaluate the propositions of the theory with bootstrapped confidence intervals and derive potential explanations for the propositions. We report on the underlying theory and the full results obtained from the replication studies with participants from 228 organisations. Our results represent a substantial step forward towards developing an empirically-based theory of RE giving insights into current practices with RE processes. The results reveal, for example, that there are no strong differences between organisations in different countries and regions, that interviews, facilitated meetings and prototyping are the most used elicitation techniques, that requirements are often documented textually, that traces between requirements and code or design documents is common, requirements specifications themselves are rarely changed and that requirements engineering (process) improvement endeavours are mostly intrinsically motivated. Our study establishes a theory that can be used as starting point for many further studies for more detailed investigations. Practitioners can use the results as theory-supported guidance on selecting suitable RE methods and techniques.

Daniel Méndez Fernández

In light of the 40th jubilee of Requirements Engineering (RE), roughly 40 experts met in Switzerland to discuss where our discipline stands today. As of today, the common view is, indisputably, that RE as a discipline is stable and respected, as pointed out by Sarah Gregory when covering the seminar in her column to which articles like this one are invited to present ongoing research. However, it is also evident that after 40 years of promising research, conducting research that industry needs is still an ongoing challenge. Research that industry needs means research that solves industrial problems practitioners face; but do we really understand those problems? Here, I want to recapitulate on this research challenge and outline an initiative, the Naming the Pain in Requirements Engineering Initiative, that aims at tackling this problem.

Daniel Méndez Fernández, Michaela Tießler, Marcos Kalinowski et al.

Background: The sensitivity of Requirements Engineering (RE) to the context makes it difficult to efficiently control problems therein, thus, hampering an effective risk management devoted to allow for early corrective or even preventive measures. Problem: There is still little empirical knowledge about context-specific RE phenomena which would be necessary for an effective context- sensitive risk management in RE. Goal: We propose and validate an evidence-based approach to assess risks in RE using cross-company data about problems, causes and effects. Research Method: We use survey data from 228 companies and build a probabilistic network that supports the forecast of context-specific RE phenomena. We implement this approach using spreadsheets to support a light-weight risk assessment. Results: Our results from an initial validation in 6 companies strengthen our confidence that the approach increases the awareness for individual risk factors in RE, and the feedback further allows for disseminating our approach into practice.

Lutz Prechelt, Daniel Graziotin, Daniel Méndez Fernández

Context: Pre-publication peer review of scientific articles is considered a key element of the research process in software engineering, yet it is often perceived as not to work fully well. Objective: We aim at understanding the perceptions of and attitudes towards peer review of authors and reviewers at one of software engineering's most prestigious venues, the International Conference on Software Engineering (ICSE). Method: We invited 932 ICSE 2014/15/16 authors and reviewers to participate in a survey with 10 closed and 9 open questions. Results: We present a multitude of results, such as: Respondents perceive only one third of all reviews to be good, yet one third as useless or misleading; they propose double-blind or zero-blind reviewing regimes for improvement; they would like to see showable proofs of (good) reviewing work be introduced; attitude change trends are weak. Conclusion: The perception of the current state of software engineering peer review is fairly negative. Also, we found hardly any trend that suggests reviewing will improve by itself over time; the community will have to make explicit efforts. Fortunately, our (mostly senior) respondents appear more open for trying different peer reviewing regimes than we had expected.

Stefan Wagner, Daniel Méndez Fernández, Michael Felderer et al.

Requirements engineering (RE) is considerably different in agile development than in more traditional development processes. Yet, there is little empirical knowledge on the state of the practice and contemporary problems in agile RE. As part of a bigger survey initiative (Naming the Pain in Requirements Engineering), we build an empirical basis on such aspects of agile RE. Based on the responses of representatives from 92 different organisations, we found that agile RE concentrates on free-text documentation of requirements elicited with a variety of techniques. Often, traces between requirements and code are explicitly managed and also software testing and RE are aligned. Furthermore, continuous improvement of RE is performed due to intrinsic motivation. Important experienced problems include unclear requirements and communication flaws. Overall, we found that most organisations conduct RE in a way we would expect and that agile RE is in several aspects not so different from RE in other development processes.

Jakob Mund, Henning Femmer, Daniel Méndez Fernández et al.

Background: Requirements Engineering is crucial for project success, and to this end, many measures for quality assurance of the software requirements specification (SRS) have been proposed. Goal: However, we still need an empirical understanding on the extent to which SRS are created and used in practice, as well as the degree to which the quality of an SRS matters to subsequent development activities. Method: We studied the relevance of SRS by relying on survey research and explored the impact of quality defects in SRS by relying on a controlled experiment. Results: Our results suggest that the relevance of SRS quality depends both on particular project characteristics and what is considered as a quality defect; for instance, the domain of safety critical systems seems to motivate for an intense usage of SRS as a means for communication whereas defects hampering the pragmatic quality do not seem to be as relevant as initially thought. Conclusion: Efficient and effective quality assurance measures must be specific for carefully characterized contexts and carefully select defect classes.

Jonas Eckhardt, Daniel Méndez Fernández, Andreas Vogelsang

Context: Seamless model-based development provides integrated chains of models, covering all software engineering phases. Non-functional requirements (NFRs), like reusability, further play a vital role in software and systems engineering, but are often neglected in research and practice. It is still unclear how to integrate NFRs in a seamless model-based development. Goal: Our long-term goal is to develop a theory on the specification of NFRs such that they can be integrated in seamless model-based development. Method: Our overall study design includes a multi-staged procedure to infer an empirically founded theory on specifying NFRs to support seamless modeling. In this short paper, we present the study design and provide a discussion of (i) preliminary results obtained from a sample, and (ii) current issues related to the design. Results: Our study already shows significant fields of improvement, e.g., the low agreement during the classification. However, the results indicate to interesting points; for example, many of commonly used NFR classes concern system modeling concepts in a way that shows how blurry the borders between functional and NFRs are. Conclusions: We conclude so far that our overall study design seems suitable to obtain the envisioned theory in the long run, but we could also show current issues that are worth discussing within the empirical software engineering community. The main goal of this contribution is not to present and discuss current results only, but to foster discussions on the issues related to the integration of NFRs in seamless modeling in general and, in particular, discussions on open methodological issues.

Marco Torchiano, Daniel Méndez Fernández, Guilherme Horta Travassos et al.

Context: Surveys constitute an valuable tool to capture a large-scale snapshot of the state of the practice. Apparently trivial to adopt, surveys hide, however, several pitfalls that might hinder rendering the result valid and, thus, useful. Goal: We aim at providing an overview of main pitfalls in software engineering surveys and report on practical ways to deal with them. Method: We build on the experiences we collected in conducting many studies and distill the main lessons learnt. Results: The eight lessons learnt we report cover different aspects of the survey process ranging from the design of initial research objectives to the design of a questionnaire. Conclusions: Our hope is that by sharing our lessons learnt, combined with a disciplined application of the general survey theory, we contribute to improving the quality of the research results achievable by employing software engineering surveys.

Marco Kuhrmann, Daniel Méndez Fernández, Thomas Ternité

Software process lines provide a systematic approach to develop and manage software processes. It defines a reference process containing general process assets, whereas a well-defined customization approach allows process engineers to create new process variants, e.g., by extending or modifying process assets. Variability operations are an instrument to realize flexibility by explicitly declaring required modifications, which are applied to create a procedurally generated company-specific process. However, little is known about which variability operations are suitable in practice. In this article, we present a study on the feasibility of variability operations to support the development of software process lines in the context of the V-Modell XT. We analyze which variability operations are defined and practically used. We provide an initial catalog of variability operations as an improvement proposal for other process models. Our findings show that 69 variability operation types are defined across several metamodel versions of which, however, 25 remain unused. The found variability operations allow for systematically modifying the content of process model elements and the process documentation, and they allow for altering the structure of a process model and its description. Furthermore, we also find that variability operations can help process engineers to compensate process metamodel evolution.

Marco Kuhrmann, Daniel Méndez Fernández

Software processes improvement (SPI) is a challenging task, as many different stakeholders, project settings, and contexts and goals need to be considered. SPI projects are often operated in a complex and volatile environment and, thus, require a sound management that is resource-intensive requiring many stakeholders to contribute to the process assessment, analysis, design, realisation, and deployment. Although there exist many valuable SPI approaches, none address the needs of both process engineers and project managers. This article presents an Artefact-based Software Process Improvement & Management approach (ArSPI) that closes this gap. ArSPI was developed and tested across several SPI projects in large organisations in Germany and Eastern Europe. The approach further encompasses a template for initiating, performing, and managing SPI projects by defining a set of 5 key artefacts and 24 support artefacts. We present ArSPI and discus results of its validation indicating ArSPI to be a helpful instrument to set up and steer SPI projects.

Marcos Kalinowski, Pablo Curty, Aline Paes et al.

[Context] Defect Causal Analysis (DCA) represents an efficient practice to improve software processes. While knowledge on cause-effect relations is helpful to support DCA, collecting cause-effect data may require significant effort and time. [Goal] We propose and evaluate a new DCA approach that uses cross-company data to support the practical application of DCA. [Method] We collected cross-company data on causes of requirements engineering problems from 74 Brazilian organizations and built a Bayesian network. Our DCA approach uses the diagnostic inference of the Bayesian network to support DCA sessions. We evaluated our approach by applying a model for technology transfer to industry and conducted three consecutive evaluations: (i) in academia, (ii) with industry representatives of the Fraunhofer Project Center at UFBA, and (iii) in an industrial case study at the Brazilian National Development Bank (BNDES). [Results] We received positive feedback in all three evaluations and the cross-company data was considered helpful for determining main causes. [Conclusions] Our results strengthen our confidence in that supporting DCA with cross-company data is promising and should be further investigated.

Daniel Méndez Fernández, Saahil Ognawala, Stefan Wagner et al.

Context: Requirements engineering process improvement (REPI) approaches have gained much attention in research and practice. Goal: So far, there is no comprehensive view on the research in REPI in terms of solutions and current state of reported evidence. We aims to provide an overview on the existing solutions, their underlying principles and their research type facets, i.e. their state of empirical evidence. Method: To this end, we conducted a systematic mapping study of the REPI publication space. Results: This paper reports on the first findings regarding research type facets of the contributions as well as selected methodological principles. We found a strong focus in the existing research on solution proposals for REPI approaches that concentrate on normative assessments and benchmarks of the RE activities rather than on holistic RE improvements according to individual goals of companies. Conclusions: We conclude, so far, that there is a need to broaden the work and to investigate more problem-driven REPI which also targets the improvement of the quality of the underlying RE artefacts, which currently seem out of scope.

Daniel Méndez Fernández, Stefan Wagner, Marcos Kalinowski et al.

As part of the Naming the Pain in Requirements Engineering (NaPiRE) initiative, researchers compared problems that companies in Brazil and Germany encountered during requirements engineering (RE). The key takeaway was that in RE, human interaction is necessary for eliciting and specifying high-quality requirements, regardless of country, project type, or company size.

Daniel Méndez Fernández, Stefan Wagner

Context: For many years, we have observed industry struggling in defining a high quality requirements engineering (RE) and researchers trying to understand industrial expectations and problems. Although we are investigating the discipline with a plethora of empirical studies, those studies either concentrate on validating specific methods or on single companies or countries. Therefore, they allow only for limited empirical generalisations. Objective: To lay an empirical and generalisable foundation about the state of the practice in RE, we aim at a series of open and reproducible surveys that allow us to steer future research in a problem-driven manner. Method: We designed a globally distributed family of surveys in joint collaborations with different researchers from different countries. The instrument is based on an initial theory inferred from available studies. As a long-term goal, the survey will be regularly replicated to manifest a clear understanding on the status quo and practical needs in RE. In this paper, we present the design of the family of surveys and first results of its start in Germany. Results: Our first results contain responses from 30 German companies. The results are not yet generalisable, but already indicate several trends and problems. For instance, a commonly stated problem respondents see in their company standards are artefacts being underrepresented, and important problems they experience in their projects are incomplete and inconsistent requirements. Conclusion: The results suggest that the survey design and instrument are well-suited to be replicated and, thereby, to create a generalisable empirical basis of RE in practice.

Marco Kuhrmann, Daniel Méndez Fernández, Jürgen Münch

Most university curricula consider software processes to be on the fringes of software engineering (SE). Students are told there exists a plethora of software processes ranging from RUP over V-shaped processes to agile methods. Furthermore, the usual students' programming tasks are of a size that either one student or a small group of students can manage the work. Comprehensive processes being essential for large companies in terms of reflecting the organization structure, coordinating teams, or interfaces to business processes such as contracting or sales, are complex and hard to teach in a lecture, and, therefore, often out of scope. We experienced tutorials on using Java or C#, or on developing applications for the iPhone to gather more attention by students, simply speaking, as these are more fun for them. So, why should students spend their time in software processes? From our experiences and the discussions with a variety of industrial partners, we learned that students often face trouble when taking their first "real" jobs, even if the company is organized in a lean or agile shape. Therefore, we propose to include software processes more explicitly into the SE curricula. We designed and implemented a course at Master's level in which students learn why software processes are necessary, and how they can be analyzed, designed, implemented, and continuously improved. In this paper, we present our course's structure, its goals, and corresponding teaching methods. We evaluate the course and further discuss our experiences so that lecturers and researchers can directly use our lessons learned in their own curricula.