D. Méndez Fernández

H. Villamizar, M. Kalinowski, M. Viana et al.

[Background] The rapidly changing business environments in which many companies operate is challenging traditional Requirements Engineering (RE) approaches. This gave rise to agile approaches for RE. Security, at the same time, is an essential non-functional requirement that still tends to be difficult to address in agile development contexts. Given the fuzzy notion of "agile" in context of RE and the difficulties of appropriately handling security requirements, the overall understanding of how to handle security requirements in agile RE is still vague. [Objective] Our goal is to characterize the publication landscape of approaches that handle security requirements in agile software projects. [Method] We conducted a systematic mapping to outline relevant work and contemporary gaps for future research. [Results] In total, we identified 21 studies that met our inclusion criteria, dated from 2005 to 2017. We found that the approaches typically involve modifying agile methods, introducing new artifacts (e.g., extending the concept of user story to abuser story), or introducing guidelines to handle security issues. We also identified limitations of using these approaches related to environment, people, effort and resources. [Conclusion] Our analysis suggests that more effort needs to be invested into empirically evaluating the existing approaches and that there is an avenue for future research in the direction of mitigating the identified limitations.

D. Méndez Fernández, W. Böhm, A. Vogelsang et al.

Artefacts play a vital role in software and systems development processes. Other terms like documents, deliverables, or work products are widely used in software development communities instead of the term artefact. In the following, we use the term `artefact' including all these other terms. Despite its relevance, the exact denotation of the term `artefact' is still not clear due to a variety of different understandings of the term and to a careless negligent usage. This often leads to approaches being grounded in a fuzzy, unclear understanding of the essential concepts involved. In fact, there does not exist a common terminology. Therefore, it is our goal that the term artefact be standardised so that researchers and practitioners have a common understanding for discussions and contributions. In this position paper, we provide a positioning and critical reflection upon the notion of artefact in software engineering at different levels of perception and how these relate to each other. We further contribute a meta model that provides a description of an artefact that is independent from any underlying process model. This meta model defines artefacts at three levels. Abstraction and refinement relations between these levels allow correlating artefacts to each other and defining the notion of related, refined, and equivalent artefacts. Our contribution shall foster the long overdue and too often underestimated terminological discussion on what artefacts are to provide a common ground with clearer concepts and principles for future software engineering contributions, such as the design of artefact-oriented development processes and tools.

X. Franch, D. Méndez Fernández, M. Oriol et al.

The relevance of Requirements Engineering (RE) research to practitioners is a prerequisite for problem-driven research in the area and key for a long-term dissemination of research results to everyday practice. To better understand how industry practitioners perceive the practical relevance of RE research, we have initiated the RE-Pract project, an international collaboration conducting an empirical study. This project opts for a replication of previous work done in two different domains and relies on survey research. To this end, we have designed a survey to be sent to several hundred industry practitioners at various companies around the world and ask them to rate their perceived practical relevance of the research described in a sample of 418 RE papers published between 2010 and 2015 at the RE, ICSE, FSE, ESEC/FSE, ESEM and REFSQ conferences. In this paper, we summarise our research protocol and present the current status of our study and the planned future steps.

M. Kuhrmann, D. Méndez Fernández, M. Daneva

Systematic literature studies have received much attention in empirical software engineering in recent years. They have become a powerful tool to collect and structure reported knowledge in a systematic and reproducible way. We distinguish systematic literature reviews to systematically analyze reported evidence in depth, and systematic mapping studies to structure a field of interest in a broader, usually quantified manner. Due to the rapidly increasing body of knowledge in software engineering, researchers who want to capture the published work in a domain often face an extensive amount of publications, which need to be screened, rated for relevance, classified, and eventually analyzed. Although there are several guidelines to conduct literature studies, they do not yet help researchers coping with the specific difficulties encountered in the practical application of these guidelines. In this article, we present an experience-based guideline to aid researchers in designing systematic literature studies with special emphasis on the data collection and selection procedures. Our guideline aims at providing a blueprint for a practical and pragmatic path through the plethora of currently available practices and deliverables capturing the dependencies among the single steps. The guideline emerges from various mapping studies and literature reviews conducted by the authors and provides recommendations for the general study design, data collection, and study selection procedures. Finally, we share our experiences and lessons learned in applying the different practices of the proposed guideline.

F. Grigoleit, A. Vetrò, D. Méndez Fernández et al.

Successful transfer of the results of research projects into practice is of great interest to all project participants. It can be assumed that different transfer mediums fulfill technology transfer (TT) with different levels of success and that they are impaired by different kinds of barriers. The goal of this study is to gain a better understanding about the different mediums used for TT in software engineering, and to identify barriers weakening the success of the application of such mediums. We conducted an exploratory study implemented by a survey in the context of a German research project with a broad range of used mediums. The main reported barriers were low expectations of usefulness, no awareness of existence, lack of resources, or inadequateness in terms of outdated material or being in an immature state. We interpreted our results as symptoms of a lack of a dissemination plan in the project. Further work will be needed to explore the implications for the transfer of research results (knowledge and techniques) to practice.

S. Wagner, D. Méndez Fernández

Most of the data produced in software projects is of textual nature: source code, specifications, or documentations. The advances in quantitative analysis methods drove a lot of data analytics in software engineering. This has overshadowed to some degree the importance of texts and their qualitative analysis. Such analysis has, however, merits for researchers and practitioners as well. In this chapter, we describe the basics of analysing text in software projects. We first describe how to manually analyse and code textual data. Next, we give an overview of mixed methods to automatic text analysis including N-Grams and clone detection as well as more sophisticated natural language processing identifying syntax and contexts of words. Those methods and tools are of critical importance to aid in the challenges in today's huge amounts of textual data. We illustrate the introduced methods via a running example and conclude by presenting two industrial studies.

M. Kalinowski, M. Felderer, T. Conte et al.

Many software projects fail due to problems in requirements engineering (RE). The goal of this paper is analyzing a specific and relevant RE problem in detail: incomplete/hidden requirements. We replicated a global family of RE surveys with representatives of software organizations in Austria and Brazil. We used the data to (a) characterize the criticality of the selected RE problem, and to (b) analyze the reported main causes and mitigation actions. Based on the analysis, we discuss how to prevent the problem. The survey includes 14 different organizations in Austria and 74 in Brazil, including small, medium and large sized companies, conducting both, plan-driven and agile development processes. Respondents from both countries cited the incomplete/hidden requirements problem as one of the most critical RE problems. We identified and graphically represented the main causes and documented solution options to address these causes. Further, we compiled a list of reported mitigation actions. From a practical point of view, this paper provides further insights into common causes of incomplete/hidden requirements and on how to prevent this problem.

D. Méndez Fernández, B. Penzenstadler

The various influences in the processes and application domains make Requirements Engineering (RE) inherently complex and difficult to implement. In general, we have two options for establishing an RE approach: we can either establish an activity-based RE approach or we can establish an artefact-based one where project participants concentrate on the RE artefacts rather than on the way of creating them. While a number of activity-based RE approaches have been proposed in recent years, we have gained much empirical evidence and experiences about the advantages of the artefact-based paradigm for RE. However, artefact orientation is still a young paradigm with various interpretations and practical manifestations whereby we need a clear understanding of its basic concepts and a consolidated and evaluated view on the paradigm. In this article, we contribute an artefact-based approach to RE (AMDiRE) that emerges from six years of experiences in fundamental and evidence-based research. To this end, we first discuss the basic notion of artefact orientation and its evolution in recent years. We briefly introduce a set of artefact-based RE models we developed in industrial research cooperations for different application domains, show their empirical evaluations, and their dissemination into academia and practice, eventually leading to the AMDiRE approach. We conclude with a discussion of experiences we made during the development and different industrial evaluations, and lessons learnt.

D. Holling, D. Méndez Fernández, A. Pretschner

Defect models capture faults and methods to provoke failures. To integrate such defect models into existing quality assurance processes, we developed a defect model lifecycle framework, in which the elicitation and classification of context-specific defects forms a crucial step. Although we could gather first insights from its practical application, we still have little knowledge about its benefits and limitations. We aim at qualitatively analyzing the context-specific elicitation and classification of defects to explore the suitability of our approach for practical application. We apply case study research in multiple contexts and analyze (1) what kind of defects we can elicit and the degree to which the defects matter to a context only, (2) the extent to which it leads to results useful enough for describing and operationalizing defect models, and (3) if there is a perceived additional immediate benefit from a practitioner's perspective. Our results strengthen our confidence on the suitability of our approach to elicit defects that are context-specific as well as context-independent. We conclude so far that our approach is suitable to provide a blueprint on how to elicit and classify defects for specific contexts to be used for the improvement of quality assurance techniques.

D. Méndez Fernández, S. Wagner, K. Lochmann et al.

Requirements Engineering (RE) is a critical discipline mostly driven by uncertainty, since it is influenced by the customer domain or by the development process model used. We aim to investigate RE processes in successful project environments to discover characteristics and strategies that allow us to elaborate RE tailoring approaches in the future. We perform a field study on a set of projects at one company. First, we investigate by content analysis which RE artefacts were produced in each project and to what extent they were produced. Second, we perform qualitative analysis of semi-structured interviews to discover project parameters that relate to the produced artefacts. Third, we use cluster analysis to infer artefact patterns and probable RE execution strategies, which are the responses to specific project parameters. Fourth, we investigate by statistical tests the effort spent in each strategy in relation to the effort spent in change requests to evaluate the efficiency of execution strategies. Our results show no statistically significant difference between the efficiency of the strategies. In addition, it turned out that many parameters considered as the main causes for project failures can be successfully handled. Hence, practitioners can apply the artefact patterns and related project parameters to tailor the RE process according to individual project characteristics.

D. Méndez Fernández, S. Wagner

For many years, we have observed industry struggling in defining a high quality requirements engineering (RE) and researchers trying to understand industrial expectations and problems. Although we are investigating the discipline with a plethora of empirical studies, they still do not allow for empirical generalisations. To lay an empirical and externally valid foundation about the state of the practice in RE, we aim at a series of open and reproducible surveys that allow us to steer future research in a problem-driven manner. We designed a globally distributed family of surveys in joint collaborations with different researchers and completed the first run in Germany. The instrument is based on a theory in the form of a set of hypotheses inferred from our experiences and available studies. We test each hypothesis in our theory and identify further candidates to extend the theory by correlation and Grounded Theory analysis. In this article, we report on the design of the family of surveys, its underlying theory, and the full results obtained from Germany with participants from 58 companies. The results reveal, for example, a tendency to improve RE via internally defined qualitative methods rather than relying on normative approaches like CMMI. We also discovered various RE problems that are statistically significant in practice. For instance, we could corroborate communication flaws or moving targets as problems in practice. Our results are not yet fully representative but already give first insights into current practices and problems in RE, and they allow us to draw lessons learnt for future replications. Our results obtained from this first run in Germany make us confident that the survey design and instrument are well-suited to be replicated and, thereby, to create a generalisable empirical basis of RE in practice.

J. Eckhardt, A. Vogelsang, D. Méndez Fernández

Non-functional requirements (NFRs) are commonly distinguished from functional requirements by differentiating how the system shall do something in contrast to what the system shall do. This distinction is not only prevalent in research, but also influences how requirements are handled in practice. NFRs are usually documented separately from functional requirements, without quantitative measures, and with relatively vague descriptions. As a result, they remain difficult to analyze and test. Several authors argue, however, that many so-called NFRs actually describe behavioral properties and may be treated the same way as functional requirements. In this paper, we empirically investigate this point of view and aim to increase our understanding on the nature of NFRs addressing system properties. We report on the classification of 530 NFRs extracted from 11 industrial requirements specifications and analyze to which extent these NFRs describe system behavior. Our results suggest that most "non-functional" requirements are not non-functional as they describe behavior of a system. Consequently, we argue that many so-called NFRs can be handled similarly to functional requirements.

H. Femmer, D. Méndez Fernández, S. Wagner et al.

Bad requirements quality can cause expensive consequences during the software development lifecycle, especially if iterations are long and feedback comes late. %-- the faster a problem is found, the cheaper it is to fix. This makes explicit the need of a lightweight detection mechanism of requirements quality violations. We aim at a light-weight static requirements analysis approach that allows for rapid checks immediately when requirements are written down. We transfer the concept of code smells to Requirements Engineering as Requirements Smells. To evaluate the benefits and limitations, we define Requirements Smells, realize our concepts for a smell detection in a prototype called Smella and apply Smella in a series of cases provided by three industrial and a university context. The automatic detection yields an average precision of 59% at an average recall of 82% with high variation. The evaluation in practical environments indicates benefits such as an increase of the awareness of quality defects. Yet, some smells were not clearly distinguishable. Lightweight smell detection can uncover many practically relevant requirements defects in a reasonably precise way. Although some smells need to be defined more clearly, smell detection provides a helpful means to support quality assurance in Requirements Engineering, for instance, as a supplement to reviews.

D. Méndez Fernández, S. Wagner

Case study research has become an important research methodology for exploring phenomena in their natural contexts. Case studies have earned a distinct role in the empirical analysis of software engineering phenomena which are difficult to capture in isolation. Such phenomena often appear in the context of methods and development processes for which it is difficult to run large, controlled experiments as they usually have to reduce the scale in several respects and, hence, are detached from the reality of industrial software development. The other side of the medal is that the realistic socio-economic environments where we conduct case studies -- with real-life cases and realistic conditions -- also pose a plethora of practical challenges to planning and conducting case studies. In this experience report, we discuss such practical challenges and the lessons we learnt in conducting case studies in industry. Our goal is to help especially inexperienced researchers facing their first case studies in industry by increasing their awareness for typical obstacles they might face and practical ways to deal with those obstacles.

P. Mafra, M. Kalinowski, D. Méndez Fernández et al.

Context] Problems in Requirements Engineering (RE) can lead to serious consequences during the software development lifecycle. [Goal] The goal of this paper is to propose empirically-based guidelines that can be used by different types of organisations according to their size (small, medium or large) and process model (agile or plan-driven) to help them in preventing such problems. [Method] We analysed data from a survey on RE problems answered by 228 organisations in 10 different countries. [Results] We identified the most critical RE problems, their causes and mitigation actions, organizing this information by clusters of size and process model. Finally, we analysed the causes and mitigation actions of the critical problems of each cluster to get further insights into how to prevent them. [Conclusions] Based on our results, we suggest preliminary guidelines for preventing critical RE problems in response to context characteristics of the companies.

J. Eckhardt, A. Vogelsang, D. Méndez Fernández

Requirements are often divided into functional requirements (FRs) and quality requirements (QRs). However, we still have little knowledge about to which extent this distinction makes sense from a practical perspective. In this paper, we report on a survey we conducted with 103 practitioners to explore whether and, if so, why they handle requirements labeled as FRs differently from those labeled as QRs. We additionally asked for consequences of this distinction w.r.t. the development process. Our results indicate that the development process for requirements of the two classes strongly differs (e.g., in testing). We identified a number of reasons why practitioners do (or do not) distinguish between QRs and FRs in their documentation and we analyzed both problems and benefits that arise from that. We found, for instance, that many reasons are based on expectations rather than on evidence. Those expectations are, in fact, not reflected in specific negative or positive consequences per se. It therefore seems more important that the decision whether to make an explicit distinction or not should be made consciously such that people are also aware of the risks that this distinction bears so that they may take appropriate countermeasures.

D. Méndez Fernández, S. Wagner, M. Kalinowski et al.

Requirements Engineering (RE) has received much attention in research and practice due to its importance to software project success. Its interdisciplinary nature, the dependency to the customer, and its inherent uncertainty still render the discipline difficult to investigate. This results in a lack of empirical data. These are necessary, however, to demonstrate which practically relevant RE problems exist and to what extent they matter. Motivated by this situation, we initiated the Naming the Pain in Requirements Engineering (NaPiRE) initiative which constitutes a globally distributed, bi-yearly replicated family of surveys on the status quo and problems in practical RE. In this article, we report on the qualitative analysis of data obtained from 228 companies working in 10 countries in various domains and we reveal which contemporary problems practitioners encounter. To this end, we analyse 21 problems derived from the literature with respect to their relevance and criticality in dependency to their context, and we complement this picture with a cause-effect analysis showing the causes and effects surrounding the most critical problems. Our results give us a better understanding of which problems exist and how they manifest themselves in practical environments. Thus, we provide a first step to ground contributions to RE on empirical observations which, until now, were dominated by conventional wisdom only.