Christoph Matthies

Franziska Dobrigkeit, Christoph Matthies, Ralf Teusner et al.

The most prominent Agile framework Scrum, is often criticized for its amount of meetings. These regular events are essential to the empirical inspect-and-adapt cycle proposed by Agile methods. Scrum meetings face several challenges, such as being perceived as boring, repetitive, or irrelevant, leading to decreased cooperation in teams and less successful projects. In an attempt to address these challenges, Agile practitioners have adopted teamwork, innovation, and design techniques geared towards improving collaboration. Additionally, they have developed their own activities to be used in Scrum meetings, most notably for conducting retrospective and planning events. Design thinking incorporates non-designers and designers in design and conceptualization activities, including user research, ideation, or testing. Accordingly, the design thinking approach provides a process with different phases and accompanying techniques for each step. These design thinking techniques can support shared understanding in teams and can improve collaboration, creativity, and product understanding. For these reasons, design thinking techniques represent a worthwhile addition to the Scrum meeting toolkit and can support Agile meetings in preventing or countering common meeting challenges and achieving meeting goals. This chapter explores how techniques from the design thinking toolkit can support Scrum meetings from a theoretical and practical viewpoint. We analyze Scrum meetings' requirements, goals, and challenges and link them to groups of techniques from the design thinking toolkit. In addition, we review interview and observational data from two previous studies with software development practitioners and derive concrete examples. As a result, we present initial guidelines on integrating design thinking techniques into Scrum meetings to make them more engaging, collaborative, and interactive.

Christoph Matthies, Franziska Dobrigkeit

Effective Retrospective meetings are vital for ensuring productive development processes because they provide the means for Agile software development teams to discuss and decide on future improvements of their collaboration. Retrospective agendas often include activities that encourage sharing ideas and motivate participants to discuss possible improvements. The outcomes of these activities steer the future directions of team dynamics and influence team happiness. However, few empirical evaluations of Retrospective activities are currently available. Additionally, most activities rely on team members experiences and neglect to take existing project data into account. With this paper we want to make a case for data-driven decision-making principles, which have largely been adopted in other business areas. Towards this goal we review existing retrospective activities and highlight activities that already use project data as well as activities that could be augmented to take advantage of additional, more subjective data sources. We conclude that data-driven decision-making principles, are advantageous, and yet underused, in modern Agile software development. Making use of project data in retrospective activities would strengthen this principle and is a viable approach as such data can support the teams in making decisions on process improvement.

Christoph Matthies, Franziska Dobrigkeit, Guenter Hesse

Software Repositories contain knowledge on how software engineering teams work, communicate, and collaborate. It can be used to develop a data-informed view of a team's development process, which in turn can be employed for process improvement initiatives. In modern, Agile development methods, process improvement takes place in Retrospective meetings, in which the last development iteration is discussed. However, previously proposed activities that take place in these meetings often do not rely on project data, instead depending solely on the perceptions of team members. We propose new Retrospective activities, based on mining the software repositories of individual teams, to complement existing approaches with more objective, data-informed process views.

Frederike Ramin, Christoph Matthies, Ralf Teusner

Motivated and competent team members are a vital part of Agile Software development and make or break any project's success. Motivation is fostered by continuous progress and recognition of efforts. These concepts are founding pillars of the Scrum methodology, which focuses on self-organizing teams. The types of contributions Scrum development team members make to a project's progress are not only technical. However, a comprehensive model comprising the varied contributions in modern software engineering teams is not yet established. We propose a model that incorporates contributions of all Scrum roles, explicitly including those which are not directly related to project artifacts. It improves the visibility of performed tasks, acts as a starting point for team retrospection, and serves as a foundation for discussion in the research community.

Christoph Matthies, Franziska Dobrigkeit

Agile methods, especially Scrum, have become staples of the modern software development industry. Retrospective meetings are Scrum's instrument for process improvement and adaptation. They are considered one of the most important aspects of the Scrum method and its implementation in organizations. However, Retrospectives face their own challenges. Agile practitioners have highlighted common problems, i.e. headaches, that repeatedly appear in meetings and negatively impact the quality of process improvement efforts. To remedy these headaches, Retrospective activities, which can help teams think together and break the usual routine, have been proposed. In this research, we present case studies of educational and industry teams, investigating the effects of eleven Retrospective activities on five identified headaches. While we find evidence for the claimed benefits of activities in the majority of studied cases, application of remedies also led to new headaches arising.

Christoph Matthies, Guenter Hesse

Software development comprises complex tasks which are performed by humans. It involves problem solving, domain understanding and communication skills as well as knowledge of a broad variety of technologies, architectures, and solution approaches. As such, software development projects include many situations where crucial decisions must be made. Making the appropriate organizational or technical choices for a given software team building a product can make the difference between project success or failure. Software development methods have introduced frameworks and sets of best practices for certain contexts, providing practitioners with established guidelines for these important choices. Current Agile methods employed in modern software development have highlighted the importance of the human factors in software development. These methods rely on short feedback loops and the self-organization of teams to enable collaborative decision making. While Agile methods stress the importance of empirical process control, i.e. relying on data to make decisions, they do not prescribe in detail how this goal should be achieved. In this paper, we describe the types and abstraction levels of data and decisions within modern software development teams and identify the benefits that usage of this data enables. We argue that the principles of data-driven decision making are highly applicable, yet underused, in modern Agile software development.

Christoph Matthies, Franziska Dobrigkeit, Guenter Hesse

Recent advances in natural-language processing and data analysis allow software bots to become virtual team members, providing an additional set of automated eyes and additional perspectives for informing and supporting teamwork. In this paper, we propose employing chatbots in the domain of software development with a focus on supporting analyses and measurements of teams' project data. The software project artifacts produced by agile teams during regular development activities, e.g. commits in a version control system, represent detailed information on how a team works and collaborates. Analyses of this data are especially relevant for agile retrospective meetings, where adaptations and improvements to the executed development process are discussed. Development teams can use these measurements to track the progress of identified improvement actions over development iterations. Chatbots provide a convenient user interface for interacting with the outcomes of retrospectives and the associated measurements in a chat-based channel that is already being employed by team members.

Christoph Matthies, Johannes Huegle, Tobias Dürschmid et al.

The perceptions and attitudes of developers impact how software projects are run and which development practices are employed in development teams. Recent agile methodologies have taken this into account, focusing on collaboration and shared team culture. In this research, we investigate the perceptions of agile development practices and their usage in Scrum software development teams. Although perceptions collected through surveys of 42 participating students did not evolve significantly over time, our analyses show that the Scrum role significantly impacted participants' views of employed development practices. We find that using the version control system according to agile ideas was consistently rated most related to the values of the Agile Manifesto. Furthermore, we investigate how common software development artifacts can be used to gain insights into team behavior and present the development data measurements we employed. We show that we can reliably detect well-defined agile practices, such Test-Driven Development, in this data and that usage of these practices coincided with participants' self-assessments.

Ralf Teusner, Christoph Matthies, Philipp Giese

In any sufficiently complex software system there are experts, having a deeper understanding of parts of the system than others. However, it is not always clear who these experts are and which particular parts of the system they can provide help with. We propose a framework to elicit the expertise of developers and recommend experts by analyzing complexity measures over time. Furthermore, teams can detect those parts of the software for which currently no, or only few experts exist and take preventive actions to keep the collective code knowledge and ownership high. We employed the developed approach at a medium-sized company. The results were evaluated with a survey, comparing the perceived and the computed expertise of developers. We show that aggregated code metrics can be used to identify experts for different software components. The identified experts were rated as acceptable candidates by developers in over 90% of all cases.

Christoph Matthies, Thomas Kowark, Keven Richly et al.

Agile methods are best taught in a hands-on fashion in realistic projects. The main challenge in doing so is to assess whether students apply the methods correctly without requiring complete supervision throughout the entire project. This paper presents experiences from a classroom project where 38 students developed a single system using a scaled version of Scrum. Surveys helped us to identify which elements of Scrum correlated most with student satisfaction or posed the biggest challenges. These insights were augmented by a team of tutors, which accompanied main meetings throughout the project to provide feedback to the teams, and captured impressions of method application in practice. Finally, we performed a post-hoc, tool-supported analysis of collaboration artifacts to detect concrete indicators for anti-patterns in Scrum adoption. Through the combination of these techniques we were able to understand how students implemented Scrum in this course and which elements require further lecturing and tutoring in future iterations. Automated analysis of collaboration artifacts proved to be a promising addition to the development process that could potentially reduce manual efforts in future courses and allow for more concrete and targeted feedback, as well as more objective assessment.

Christoph Matthies, Thomas Kowark, Keven Richly et al.

Linting tools automatically identify source code fragments that do not follow a set of predefined standards. Such feedback tools are equally desirable for "linting" agile development processes. However, providing concrete feedback on process conformance is a challenging task, due to the intentional lack of formal agile process models. In this paper, we present ScrumLint, a tool that tackles this issue by analyzing development artifacts. On the basis of experiences with an undergraduate agile software engineering course, we defined a collection of process metrics. These contain the core ideas of agile methods and report deviations. Using this approach, development teams receive immediate feedback on their executed development practices. They can use this knowledge to improve their workflows, or can adapt the metrics to better reflect their project reality.

Christoph Matthies, Thomas Kowark, Matthias Uflacker et al.

Teaching agile software development by pairing lectures with hands-on projects has become the norm. This approach poses the problem of grading and evaluating practical project work as well as process conformance during development. Yet, few best practices exist for measuring the success of students in implementing agile practices. Most university courses rely on observations during the course or final oral exams. In this paper, we propose a set of metrics which give insights into the adherence to agile practices in teams. The metrics identify instances in development data, e.g. commits or user stories, where agile processes were not followed. The identified violations can serve as starting points for further investigation and team discussions. With contextual knowledge of the violation, the executed process or the metric itself can be refined. The metrics reflect our experiences with running a software engineering course over the last five years. They measure aspects which students frequently have issues with and that diminish process adoption and student engagement. We present the proposed metrics, which were tested in the latest course installment, alongside tutoring, lectures, and oral exams.

Christoph Matthies, Arian Treffer, Matthias Uflacker

Teaching programming using Massive Open Online Courses (MOOCs) is gaining popularity due to their scalability and efficiency of knowledge distribution. However, participating in these courses usually means fully committing to the supplied programming environment in the browser. While this allows a consistent and controllable setup, learners do not gain experience with actual development tools, such as local code editors, testing frameworks, issue trackers or continuous integration (CI) services, which is critical for subsequent real-world projects. Furthermore, the tests for the functionality that is to be developed are oftentimes already available in MOOCs and simply need to be executed, leading to less involvement with developing appropriate tests. In order to tackle these issues while maintaining a high degree of automation and scalability, we developed Prof. CI, a novel approach to conducting online exercises. Prof. CI leverages the existing automation infrastructure that developers use daily, i.e. CI services and Github workflows, to teach test-driven development (TDD) practices. Participants work on their own repositories in Github and receive feedback and new challenges from the CI server when they push their code. We have successfully applied this approach in a pilot project with 30 undergraduate students learning the Ruby on Rails web development framework. Our evaluation shows that the exercise effectively increased students' motivation to write tests for their code. We also present the results of participant surveys, students' experiences and teachers' observations.

Christoph Matthies, Ralf Teusner, Guenter Hesse

This Innovative Practice Full Paper presents an approach of using software development artifacts to gauge student behavior and the effectiveness of changes to curriculum design. There is an ongoing need to adapt university courses to changing requirements and shifts in industry. As an educator it is therefore vital to have access to methods, with which to ascertain the effects of curriculum design changes. In this paper, we present our approach of analyzing software repositories in order to gauge student behavior during project work. We evaluate this approach in a case study of a university undergraduate software development course teaching agile development methodologies. Surveys revealed positive attitudes towards the course and the change of employed development methodology from Scrum to Kanban. However, surveys were not usable to ascertain the degree to which students had adapted their workflows and whether they had done so in accordance with course goals. Therefore, we analyzed students' software repository data, which represents information that can be collected by educators to reveal insights into learning successes and detailed student behavior. We analyze the software repositories created during the last five courses, and evaluate differences in workflows between Kanban and Scrum usage.

Ralf Teusner, Christoph Matthies, Thomas Staubitz

This work presents insights about the long-term effects and retention rates of knowledge acquired within MOOCs. In 2015 and 2017, we conducted two introductory MOOCs on object-oriented programming in Java with each over 10,000 registered participants. In this paper, we analyze course scores, quiz results and self-stated skill levels of our participants. The aim of our analysis is to uncover factors influencing the retention of acquired knowledge, such as time passed or knowledge application, in order to improve long-term success. While we know that some participants learned the programming basics within our course, we lack information on whether this knowledge was applied and fortified after the course's end. To fill this knowledge gap, we conducted a survey in 2018 among all participants of our 2015 and 2017 programming MOOCs. The first part of the survey elicits responses on whether and how MOOC knowledge was applied and gives participants opportunity to voice individual feedback. The second part of the survey contains several questions of increasing difficulty and complexity regarding course content in order to learn about the consolidation of the acquired knowledge. We distinguish three programming knowledge areas in the survey: First, understanding of concepts, such as loops and boolean algebra. Second, syntax knowledge, such as specific keywords. Third, practical skills including debugging and coding. We further analyzed the long-term effects separately per participant skill group. While answer rates were low, the collected data shows a decrease of knowledge over time, relatively unaffected by skill level. Application of the acquired knowledge improves the memory retention rates of MOOC participants across all skill levels.

Christoph Matthies

Using university capstone courses to teach agile software development methodologies has become commonplace, as agile methods have gained support in professional software development. This usually means students are introduced to and work with the currently most popular agile methodology: Scrum. However, as the agile methods employed in the industry change and are adapted to different contexts, university courses must follow suit. A prime example of this is the Kanban method, which has recently gathered attention in the industry. In this paper, we describe a capstone course design, which adds the hands-on learning of the lean principles advocated by Kanban into a capstone project run with Scrum. This both ensures that students are aware of recent process frameworks and ideas as well as gain a more thorough overview of how agile methods can be employed in practice. We describe the details of the course and analyze the participating students' perceptions as well as our observations. We analyze the development artifacts, created by students during the course in respect to the two different development methodologies. We further present a summary of the lessons learned as well as recommendations for future similar courses. The survey conducted at the end of the course revealed an overwhelmingly positive attitude of students towards the integration of Kanban into the course.