Lucas Gren

Emanuel Mellblom, Isar Arason, Lucas Gren et al.

This paper examines the connection between the Five Factor Model personality traits and burnout in software developers. This study aims to validate generalizations of findings in other fields. An online survey consisting of a miniaturized International Personality Item Pool questionnaire for measuring the Five Factor Model personality traits, and the Shirom-Melamed Burnout Measure for measuring burnout, were distributed to open source developer mailing lists, obtaining 47 valid responses. The results from a Bayesian Linear Regression analysis indicate a strong link between neuroticism and burnout confirming previous work, while the other Five Factor Model traits were not adding power to the model. It is important to note that we did not investigate the quality of work in connection to personality, nor did we take any other confounding factors into account like, for example, teamwork. Nonetheless, employers could be aware of, and support, software developers with high neuroticism.

Lucas Gren, Paul Ralph

Effective leadership is one of the key drivers of business and project success, and one of the most active areas of management research. But how does leadership work in agile software development, which emphasizes self-management and self-organization and marginalizes traditional leadership roles? To find out, this study examines agile leadership from the perspective of thirteen professionals who identify as agile leaders, in different roles, at ten different software development companies of varying sizes. Data from semi-structured interviews reveals that leadership: (1) is dynamically shared among team members; (2) engenders a sense of belonging to the team; and (3) involves balancing competing organizational cultures (e.g. balancing the new agile culture with the old milestone-driven culture). In other words, agile leadership is a property of a team, not a role, and effectiveness depends on agile team members' identifying with the team, accepting responsibility, and being sensitive to cultural conflict.

Lucas Gren, Richard Berntsson Svensson

Context: Expert judgement is a common method for software effort estimations in practice today. Estimators are often shown extra obsolete requirements together with the real ones to be implemented. Only one previous study has been conducted on if such practices bias the estimations. Objective: We conducted six experiments with both students and practitioners to study, and quantify, the effects of obsolete requirements on software estimation. Method By conducting a family of six experiments using both students and practitioners as research subjects (N = 461), and by using a Bayesian Data Analysis approach, we investigated different aspects of this effect. We also argue for, and show an example of, how we by using a Bayesian approach can be more confident in our results and enable further studies with small sample sizes. Results: We found that the presence of obsolete requirements triggered an overestimation in effort across all experiments. The effect, however, was smaller in a field setting compared to using students as subjects. Still, the over-estimations triggered by the obsolete requirements were systematically around twice the percentage of the included obsolete ones, but with a large 95% credible interval. Conclusions: The results have implications for both research and practice in that the found systematic error should be accounted for in both studies on software estimation and, maybe more importantly, in estimation practices to avoid over-estimation due to this systematic error. We partly explain this error to be stemming from the cognitive bias of anchoring-and-adjustment, i.e. the obsolete requirements anchored a much larger software. However, further studies are needed in order to accurately predict this effect.

Lucas Gren

The purpose of this paper is to suggest additional aspects of social psychology that could help when making sense of autonomous agile teams. To make use of well-tested theories in social psychology and instead see how they replicated and differ in the autonomous agile team context would avoid reinventing the wheel. This was done, as an initial step, through looking at some very common agile practices and relate them to existing findings in social-psychological research. The two theories found that I argue could be more applied to the software engineering context are social identity theory and group socialization theory. The results show that literature provides social-psychological reasons for the popularity of some agile practices, but that scientific studies are needed to gather empirical evidence on these under-researched topics. Understanding deeper psychological theories could provide a better understanding of the psychological processes when building autonomous agile team, which could then lead to better predictability and intervention in relation to human factors.

Lucas Gren, Magdalena Lindman

When large industrial organizations change to(or start with) an agile approach to operations, managers and some employees are supposed to be "agile leaders" often without being given a clear definition of what that comprises when building agile teams. An inductive thematic analysis was used to investigate what 15 appointed leaders actually do and perceive as challenges regarding group dynamics working with an agile approach. Team maturity, Team design, and Culture and mindset were all categories of challenges related to group dynamics that the practitioners face and manage in their work-life that are not explicitly mentioned in the more process-focused agile transformation frameworks. The results suggest that leader mitigation of these three aspects of group dynamics is essential to the success of an agile transformation.

Lucas Gren

Contribution: A flipped classroom approach to teaching empirical software engineering increases student learning by providing more time for active learning in class. Background: There is a need for longitudinal studies of the flipped classroom approach in general. Although a few cross-sectional studies show that a flipped classroom approach can increase student learning by providing more time for other in-class activities, such as active learning, such studies are also rare in the context of teaching software engineering. Intended outcomes: To assess the usefulness of a flipped classroom approach in teaching software engineering. Application design: The study was conducted at an international Master's program in Sweden, given in English, and partially replicated at a university in Africa. Findings: The results suggest that students' academic success, as measured by their exam grades, can be improved by introducing a flipped classroom to teach software engineering topics, but this may not extend to their subjective liking of the material, as measured by student evaluations. Furthermore, the effect of the change in teaching methodology was not replicated when changing the teaching team.

Lucas Gren, Alfredo Goldman, Christian Jacobsson

With the agile approach to managing software development projects comes an increased dependability on well functioning teams, since many of the practices are built on teamwork. The objective of this study was to investigate if, and how, team development from a group psychological perspective is related to some work practices of agile teams. Data were collected from 34 agile teams (200 individuals) from six software development organizations and one university in both Brazil and Sweden using the Group Development Questionnaire (Scale IV) and the Perceptive Agile Measurement (PAM). The result indicates a strong correlation between levels of group maturity and the two agile practices \emph{iterative development} and \emph{retrospectives}. We, therefore, conclude that agile teams at different group development stages adopt parts of team agility differently, thus confirming previous studies but with more data and by investigating concrete and applied agile practices. We thereby add evidence to the hypothesis that an agile implementation and management of agile projects need to be adapted to the group maturity levels of the agile teams.

Lucas Gren, Alfredo Goldman, Christian Jacobsson

Research has shown that the maturity of small workgroups from a psychological perspective is intimately connected to team agility. We, therefore, tested if agile team members appreciated group development psychology training. Our results show that the participating teams seem to have a very positive view of group development training and state that they now have a new way of thinking about teamwork and new tools to deal with team-related problems. We, therefore, see huge potential in training agile teams in group development psychology since the positive effects might span over the entire software development organization.

Lucas Gren, Per Lenberg

There is some ambiguity of what agile means in both research and practice. Authors have suggested a diversity of different definitions, through which it is difficult to interpret what agile really is. The concept, however, exists in its implementation through agile practices. In this vision paper, we argue that adopting an agile approach boils down to being more responsive to change. To support this claim, we relate agile principles, practices, the agile manifesto, and our own experiences to this core definition. We envision that agile transformations would be, and are, much easier using this definition and contextualizing its implications.

Mayra Nilson, Vard Antinyan, Lucas Gren

Internal software quality determines the maintainability of the software product and influences the quality in use. There is a plethora of metrics which purport to measure the internal quality of software, and these metrics are offered by static software analysis tools. To date, a number of reports have assessed the validity of these metrics. No data are available, however, on whether metrics offered by the tools are somehow validated in scientific studies. The current study covers this gap by providing data on which tools and how many validated metrics are provided. The results show that a range of metrics that the tools provided do not seem to be validated in the literature and that only a small percentage of metrics are validated in the provided tools.

Lucas Gren, Alexander Wong, Erik Kristoffersson

Agile methods have gotten a good reputation for managing projects in many different sectors. A challenge among practitioners in the ERP (Enterprise Resource Planning) domain, is to decide if an agile method is suitable or not for new projects. This study investigates how decision-makers select between agile and plan-driven ERP implementation projects in relation to known factors from previous research. We selected projects that the decision-makers assessed as successful, but borderline, agile and plan-driven projects already implemented and let project managers fill out a survey consisting of key agile or plan-driven characteristics. We found that the assessment made by decision-makers did not differ on any aspects except higher executive buy-in for process change, and the prioritization of low cost in the agile projects. This study highlights the difficulty in selecting implementation strategy for a large part of the ERP implementations, and the data show that the decision-makers could not make such a decision at an early point in time without contextual knowledge.

Lucas Gren

An agile change effort in an organization needs to be understood in relation to human processes. Such theory and accompanying tools already existed almost 50 years ago in applied psychology. The core ideas of Agile Process Consultation are that a client initiating a change toward more agility often does not know what is wrong and the consultant needs to diagnose the problem jointly with the client. It is also assumed that the agile consultant cannot know the organizational culture of the client's organization, which means that the client needs to be making the decisions based on the suggestions provided by the consultant. Since agile project management is spreading across the enterprise and outside of software development, we need situational approaches instead of prescribing low-level practices.

Lucas Gren, Per Lenberg

Today, software companies usually organize their work in teams. Social science research on team development has shown that for a team to reach a productive and autonomous stage, it has to be able to manage internal conflicts and disagreements efficiently. To better facilitate the team development process, we argue that software engineers' needs additional training in negotiation skills and conflict resolution. In this position paper, we outline ideas for what aspects to consider in such training. As an example, we argue that a majority of the conflicts originate from team-level factors and that they, therefore, should be managed on the team-level instead of in relation to dyads.

Lucas Gren

The purpose of this paper is to review research on attitudes and social norms and connect it to the agile software development context. Furthermore, I propose additional theories from social psychology (mainly the theory of planned behavior and using the degree of internalization of social norms) that would most certainly be useful for further sense-making of human factors-related research on agile teams.

Lucas Gren

This note highlights the importance of investigating diversity aspects in combination in empirical research. It draws on the psychological discourse and suggests why and how software engineering scholars can use the aspect of diversity in all their research endeavors.

Khaled Al-Sabbagh, Lucas Gren

Empirical evidence regarding the connection between group development (maturity) and the success of software development teams is lacking. The purpose of this research is to gain a qualitative and quantitative understanding of how velocity and planning effectiveness of software teams connect to a group development model. The Group Development Questionnaire was given to 19 software developers from 4 work groups to assess their group development maturity. The work groups' responses to the survey were checked for correlation with development velocity and planning effectiveness. Semistructured interviews were conducted with 16 individuals from the same 4 work groups to explore issues about their group maturity and to validate the responses of the interviewees in the Group Development Questionnaire. The measurement of the fourth stage of group development had a strong association with the planning effectiveness measurement, which means that a team with less issues in the fourth phase of group development is more effective in adhering to its plans. On the other hand, group development and velocity showed no significant convergent validity. We conclude that the dynamics within software development teams might correlate to their ability to deliver the expected outcome as planned but not to their ability to develop tasks faster.

Lucas Gren

Scholars have tried to explain how organizations can build agile teams by only looking at one level of analysis. We argue in this short paper that lessons can be learned from organizational science results explaining variance on three different abstraction levels of organizations. We suggest agility needs to be explained from organizational (macro), the team (meso), and individual (micro) levels to provide useful and actionable guidelines to practitioners. We are currently designing such studies and hope that they will eventually result in validated measurements that can be used to prevent companies from investing in the wrong areas when trying to move towards more agility.

Lucas Gren

Agile requirements engineering implies more complex communication patterns since even the developers are supposed to have direct contact with customers. With more face-to-face communication comes social-psychological factors influencing the requirements. Studies have pointed at the importance of negotiation training, but I argue that more basic human traits can be triggered in favor of the negotiator with the most knowledge of social influence research and practice. I suggest a plan of how research in social influence and requirements facilitation can be conducted, mostly through experimentation.

Lucas Gren, Vard Antinyan

Unit testing has been considered as having a key role in building high quality software, and therefore it has been widely used in practice. However, data on the relationship between unit testing and aspects of software quality remain scarce. A survey study with 235 survey responses from seven organizations was conducted in order to understand the correlation between practitioners' perception of code quality and unit testing practices. In addition, we conducted a case study in one of these organizations to investigate the correlation between unit test coverage and post-unit test defects. In both cases none or weak correlations were found. We recommend further research on the effectiveness of different testing practices in order to help practitioners to understand how to best allocate their resources to the testing chain.

Lucas Gren

Agile processes explicitly focus more on team-work than more traditional management techniques when building software. With high velocity and responsiveness on team-level come the risk of interpersonal conflict in the agile organizations. Through a survey with 68 software developers from three large Swedish companies, I found that the presence of interpersonal conflict was negatively connected to the agile practices Iterative Development and Customer Access. The agile practices Iteration Planning and Iterative Development were positively linked to the measurement of the developers' perceived team productivity. However, Continuous Integration & Testing was negatively connected to productivity. These results show which agile practices are directly linked to team productivity, but also, and more importantly, indicate which of the agile practices that might be more prone to not work as intended, when the team struggles with interpersonal conflict. Therefore, I argue that members of agile teams need training in conflict resolution techniques in order to lower the risk of interpersonal conflict negatively affecting team productivity.

Lucas Gren

Agility implies a set of principles that need to be followed in order to have the proposed responsiveness to change. This paper presents how the Agile Adoption Framework can be used to assess agility and pinpoint focus areas for companies that want to improve. Management found it useful in general to get data on possible focus points for improvement in their agile transformation. We believe the suggested tool could be useful as one step in the agile process assessment.

Lucas Gren, Richard Berntsson Svensson, Michael Unterkalmsteiner

Effort estimation is a complex area in decision-making, and is influenced by a diversity of factors that could increase the estimation error. The effects on effort estimation accuracy of having obsolete requirements in specifications have not yet been studied. This study aims at filling that gap. A total of 150 students were asked to provide effort estimates for different amounts of requirements, and one group was explicitly told to disregard some of the given requirements. The results show that even the extra text instructing participants to exclude requirements in the estimation task, had the subjects give higher estimates. The effect of having obsolete requirements in requirements specifications and backlogs in software effort estimation is not taken into account enough today, and this study provides empirical evidence that it possibly should. We also suggest different psychological explanations to the found effect.

Lucas Gren

Brooks' Law is often referred to in practice and states that adding manpower to a late software project makes it even later. Brooks' himself gave three explanation only related to concrete task-related issues, like introducing new members to the work being done, communication overheads, or difficulty dividing some programming tasks. Through a description of group developmental psychology we argue for a fourth explanation to the law by suggesting that the group will fall back in its group development when new members are added, resulting in rework setting group norms, group goals, defining roles etc. that will also change over time. We show that this fourth explanation is important when trying to understanding Brooks' Law, and that adding the group developmental perspective might help software development organizations in managing projects.

Lucas Gren, Richard Torkar, Robert Feldt

The agile approach to projects focuses more on close-knit teams than traditional waterfall projects, which means that aspects of group maturity become even more important. This psychological aspect is not much researched in connection to the building of an "agile team." The purpose of this study is to investigate how building agile teams is connected to a group development model taken from social psychology. We conducted ten semi-structured interviews with coaches, Scrum Masters, and managers responsible for the agile process from seven different companies, and collected survey data from 66 group-members from four companies (a total of eight different companies). The survey included an agile measurement tool and the one part of the Group Development Questionnaire. The results show that the practitioners define group developmental aspects as key factors to a successful agile transition. Also, the quantitative measurement of agility was significantly correlated to the group maturity measurement. We conclude that adding these psychological aspects to the description of the "agile team" could increase the understanding of agility and partly help define an "agile team." We propose that future work should develop specific guidelines for how software development teams at different maturity levels might adopt agile principles and practices differently.

Lucas Gren, Alfredo Goldman

There has been some evidence that agility is connected to the group maturity of software development teams. This study aims at conducting group development psychology training with student teams, participating in a project course at university, and compare their group effectiveness score to their agility usage over time in a longitudinal design. Seven XP student teams were measured twice (43+40), which means 83 data points divided into two groups (an experimental group and one control group). The results showed that the agility measurement was not possible to increase by giving a 1.5-hour of group psychology lecture and discussion over a two-month period. The non-significant result was probably due to the fact that 1.5 hours of training were not enough to change the work methods of these student teams, or, a causal relationship does not exist between the two concepts. A third option could be that the experiential setting of real teams, even at a university, has many more variables not taken into account in this experiment that affect the two concepts. We therefore have no conclusions to draw based on the expected effects. However, we believe these concepts have to be connected since agile software development is based on teamwork to a large extent, but there are probably many more confounding or mediating factors.

Lucas Gren

There is a diversity of models explaining organizational culture and how these complex aspects can be addressed in connection to organizational change efforts. This workshop paper claims that models already exist for dealing with the cultural change that an agile transition is in the software engineering context. Instead of realizing this again through agile success stories, and thus reinventing the wheel, it is argued that the research in the software engineering field should build on these models instead and investigate how/if they differ. Practitioners already work as the change agents described in other fields and they should get recognition through the presence and integration of these models in the software engineering process research.

Konstantinos Chronis, Lucas Gren

Many tools have been created for measuring the agility of software teams, thus creating a saturation in the field. Three agile measurement tools were selected in order to validate whether they yield sim-ilar results. The surveys of the tools were given to teams in Company A (N = 30). The questions were grouped into agile practices which were checked for correlation in order to establish convergent validity. In addition, we checked whether the questions identified to be the same among the tools would be given the same replies by the respondents. We could not establish convergent validity since the correlations of the data gathered were very few and low. In addition, the questions which were identified to have the same meaning among the tools did not have the same answers from the respondents. We conclude that the area of measuring agility is still immature and more work needs to be done. Not all tools are applicable to every team but they should be selected on the basis of how a team has transitioned to agile.

Lucas Gren, Alfredo Goldman

In this paper we describe the usefulness of statistical validation techniques for human factors survey research. We need to investigate a diversity of validity aspects when creating metrics in human factors research, and we argue that the statistical tests used in other fields to get support for reliability and construct validity in surveys, should also be applied to human factors research in software engineering more often. We also show briefly how such methods can be applied (Test-Retest, Cronbach's α, and Exploratory Factor Analysis).

Lucas Gren, Richard Torkar, Robert Feldt

The focus on psychology has increased within software engineering due to the project management innovation "agile development processes". The agile methods do not explicitly consider group development aspects; they simply assume what is described in group psychology as mature groups. This study was conducted with 45 employees and their twelve managers (N=57) from two SAP customers in the US that were working with agile methods, and the data were collected via an online survey. The selected Agility measurement was correlated to a Group Development measurement and showed significant convergent validity, i.e., a more mature team is also a more agile team. This means that the agile methods probably would benefit from taking group development into account when its practices are being introduced.

Lucas Gren, Richard Torkar, Robert Feldt

Agile development has now become a well-known approach to collaboration in professional work life. Both researchers and practitioners want validated tools to measure agility. This study sets out to validate an agile maturity measurement model with statistical tests and empirical data. First, a pretest was conducted as a case study including a survey and focus group. Second, the main study was conducted with 45 employees from two SAP customers in the US. We used internal consistency (by a Cronbach's alpha) as the main measure for reliability and analyzed construct validity by exploratory principal factor analysis (PFA). The results suggest a new categorization of a subset of items existing in the tool and provides empirical support for these new groups of factors. However, we argue that more work is needed to reach the point where a maturity models with quantitative data can be said to validly measure agility, and even then, such a measurement still needs to include some deeper analysis with cultural and contextual items.

Lucas Gren, Richard Torkar, Robert Feldt

There are studies showing what happens if agile teams are introduced into a non-agile organization, e.g. higher overhead costs and the necessity of an understanding of agile methods even outside the teams. This case study shows an example of work motivational aspects that might surface when an agile team exists in the middle of a more traditional structure. This case study was conducted at a car manufacturer in Sweden, consisting of an unstructured interview with the Scrum Master and a semi-structured focus group. The results show that the teams felt that the feedback from the surrounding organization was unsynchronized resulting in them not feeling appreciated when delivering their work. Moreover, they felt frustrated when working on non-agile teams after have been working on agile ones. This study concludes that there were work motivational affects of fitting an agile team into a non-agile surrounding organization, and therefore this might also be true for other organizations.

Andreas Bäckevik, Erik Tholén, Lucas Gren

An agile approach has become very popular over the last decade, which requires good communication and teamwork within teams as well as with outside stakeholders. Therefore, social interaction is central for a software development team to be successful. Such social interactions form social identities and social structures in both teams and organizations. This study investigates possible effects that the social identity of individuals may have on the effectiveness of software development through seven in-dept interviews. The qualitative data from interviews were analyzed and summarized using summative content analysis, and the seven individuals also answered a questionnaire on social identity taken from social psychology research. The qualitative result shows that aspects of social identity affect software developers' behavior, and that we need to build cross-functional stable teams over time also from a pure social identity perspective in addition to the product related aspects to avoid a decreased effectiveness. However, we did not see clear connections to our operationalization of effectiveness in this study, and the quantitative analysis was also inconclusive, but we see value in our suggested method when investigating social identity in software development.

Lucas Gren, Per Lenberg, Karolina Ljungberg

Social psychology researchers have, traditionally, focused on the construct of thinking rather than on feeling. Since the beginning of the 21st century, social science researchers have, however, increasingly explored the effects of affect. Their work has repeatedly recognized that affects play a crucial role in determining people's behavior. In this short paper, we argue that software engineering studies on affect would benefit from using more of the knowledge that social science researchers have acquired. Without accounting for their findings, we risk re-inventing the wheel. Also, without a profound understanding of the complex interplay between social context and affect, we risk creating overly simplistic solutions that might have considerable long-term adverse effects for software engineers.

Richard Torkar, Carlo A. Furia, Robert Feldt et al.

A key goal of empirical research in software engineering is to assess practical significance, which answers whether the observed effects of some compared treatments show a relevant difference in practice in realistic scenarios. Even though plenty of standard techniques exist to assess statistical significance, connecting it to practical significance is not straightforward or routinely done; indeed, only a few empirical studies in software engineering assess practical significance in a principled and systematic way. In this paper, we argue that Bayesian data analysis provides suitable tools to assess practical significance rigorously. We demonstrate our claims in a case study comparing different test techniques. The case study's data was previously analyzed (Afzal et al., 2015) using standard techniques focusing on statistical significance. Here, we build a multilevel model of the same data, which we fit and validate using Bayesian techniques. Our method is to apply cumulative prospect theory on top of the statistical model to quantitatively connect our statistical analysis output to a practically meaningful context. This is then the basis both for assessing and arguing for practical significance. Our study demonstrates that Bayesian analysis provides a technically rigorous yet practical framework for empirical software engineering. A substantial side effect is that any uncertainty in the underlying data will be propagated through the statistical model, and its effects on practical significance are made clear. Thus, in combination with cumulative prospect theory, Bayesian analysis supports seamlessly assessing practical significance in an empirical software engineering context, thus potentially clarifying and extending the relevance of research for practitioners.

Lucas Gren, Alessia Knauss, Christoph Johann Stettina

Context: Existing knowledge in agile software development suggests that individual competency (e.g. skills) is a critical success factor for agile projects. While assuming that technical skills are important for every kind of software development project, many researchers suggest that non-technical individual skills are especially important in agile software development. Objective: In this paper, we investigate whether non-technical individual skills can predict the use of agile practices. Method: Through creating a set of multiple linear regression models using a total of 113 participants from agile teams in six software development organizations from The Netherlands and Brazil, we analyzed the predictive power of non-technical individual skills in relation to agile practices. Results: The results show that there is surprisingly low power in using non-technical individual skills to predict (i.e. explain variance in) the mature use of agile practices in software development. Conclusions: Therefore, we conclude that looking at non-technical individual skills is not the optimal level of analysis when trying to understand, and explain, the mature use of agile practices in the software development context. We argue that it is more important to focus on the non-technical skills as a team-level capacity instead of assuring that all individuals possess such skills when understanding the use of the agile practices.

Lucas Gren

Background. There are some publications in software engineering research that aim at guiding researchers in assessing validity threats to their studies. Still, many researchers fail to address many aspects of validity that are essential to quantitative research on human factors. Goal. This paper has the goal of triggering a change of mindset in what types of studies are the most valuable to the behavioral software engineering field, and also provide more details of what construct validity is. Method. The approach is based on psychological test theory and draws upon methods used in psychology in relation to construct validity. Results. In this paper, I suggest a different approach to validity threats than what is commonplace in behavioral software engineering research. Conclusions. While this paper focuses on behavioral software engineering, I believe other types of software engineering research might also benefit from an increased focus on construct validity.

Per Lenberg, Robert Feldt, Lucas Gren et al.

Researchers are increasingly recognizing the importance of human aspects in software development. Since qualitative methods are used to explore human behavior in-depth, we believe that studies using such methods will become more common. Existing qualitative software engineering guidelines do not cover the full breadth of qualitative methods and the knowledge on how to use them like in social sciences. The purpose of this study was to extend the software engineering community's current body of knowledge regarding available qualitative methods and their quality assurance frameworks, and to provide recommendations and guidelines for their use. With the support of an epistemological argument and a survey of the literature, we suggest that future research would benefit from (1) utilizing a broader set of research methods, (2) more strongly emphasizing reflexivity, and (3) employing qualitative guidelines and quality criteria. We present an overview of three qualitative methods commonly used in social sciences but rarely seen in software engineering research, namely interpretative phenomenological analysis, narrative analysis, and discourse analysis. Furthermore, we discuss the meaning of reflexivity in relation to the software engineering context and suggest means of fostering it. Our paper will help software engineering researchers better select and then guide the application of a broader set of qualitative research methods.

Francisco Gomes de Oliveira Neto, Richard Torkar, Robert Feldt et al.

Software engineering research is evolving and papers are increasingly based on empirical data from a multitude of sources, using statistical tests to determine if and to what degree empirical evidence supports their hypotheses. To investigate the practices and trends of statistical analysis in empirical software engineering (ESE), this paper presents a review of a large pool of papers from top-ranked software engineering journals. First, we manually reviewed 161 papers and in the second phase of our method, we conducted a more extensive semi-automatic classification of papers spanning the years 2001--2015 and 5,196 papers. Results from both review steps was used to: i) identify and analyze the predominant practices in ESE (e.g., using t-test or ANOVA), as well as relevant trends in usage of specific statistical methods (e.g., nonparametric tests and effect size measures) and, ii) develop a conceptual model for a statistical analysis workflow with suggestions on how to apply different statistical methods as well as guidelines to avoid pitfalls. Lastly, we confirm existing claims that current ESE practices lack a standard to report practical significance of results. We illustrate how practical significance can be discussed in terms of both the statistical analysis and in the practitioner's context.