Uirá Kulesza

João Helis Bernardo, Daniel Alencar da Costa, Sérgio Queiroz de Medeiros et al.

Continuous Integration (CI) is a well-established practice in traditional software development, but its nuances in the domain of Machine Learning (ML) projects remain relatively unexplored. Given the distinctive nature of ML development, understanding how CI practices are adopted in this context is crucial for tailoring effective approaches. In this study, we conduct a comprehensive analysis of 185 open-source projects on GitHub (93 ML and 92 non-ML projects). Our investigation comprises both quantitative and qualitative dimensions, aiming to uncover differences in CI adoption between ML and non-ML projects. Our findings indicate that ML projects often require longer build durations, and medium-sized ML projects exhibit lower test coverage compared to non-ML projects. Moreover, small and medium-sized ML projects show a higher prevalence of increasing build duration trends compared to their non-ML counterparts. Additionally, our qualitative analysis illuminates the discussions around CI in both ML and non-ML projects, encompassing themes like CI Build Execution and Status, CI Testing, and CI Infrastructure. These insights shed light on the unique challenges faced by ML projects in adopting CI practices effectively.

João Helis Bernardo, Daniel Alencar da Costa, Filipe Roseiro Cogo et al.

Continuous Integration (CI) is a cornerstone of modern software development. However, while widely adopted in traditional software projects, applying CI practices to Machine Learning (ML) projects presents distinctive characteristics. For example, our previous work revealed that ML projects often experience longer build durations and lower test coverage rates compared to their non-ML counterparts. Building on these quantitative findings, this study surveys 155 practitioners from 47 ML projects to investigate the underlying reasons for these distinctive characteristics through a qualitative perspective. Practitioners highlighted eight key differences, including test complexity, infrastructure requirements, and build duration and stability. Common challenges mentioned by practitioners include higher project complexity, model training demands, extensive data handling, increased computational resource needs, and dependency management, all contributing to extended build durations. Furthermore, ML systems' non-deterministic nature, data dependencies, and computational constraints were identified as significant barriers to effective testing. The key takeaway from this study is that while foundational CI principles remain valuable, ML projects require tailored approaches to address their unique challenges. To bridge this gap, we propose a set of ML-specific CI practices, including tracking model performance metrics and prioritizing test execution within CI pipelines. Additionally, our findings highlight the importance of fostering interdisciplinary collaboration to strengthen the testing culture in ML projects. By bridging quantitative findings with practitioners' insights, this study provides a deeper understanding of the interplay between CI practices and the unique demands of ML projects, laying the groundwork for more efficient and robust CI strategies in this domain.

Rodrigo Rebouças de Almeida, Christoph Treude, Uirá Kulesza

What are the business causes behind tight deadlines? What drives the prioritization of features that pushes quality matters to the back burner? We conducted a survey with 71 experienced practitioners and did a thematic analysis of the open-ended answers to the question: ``Could you give examples of how business may contribute to technical debt?'' Business-related causes were organized into two categories: pure-business and business/IT gap, and they were related to `tight deadlines' and `features over quality', the most frequently cited management reasons for technical debt. We contribute a cause-effect model which relates the various business causes of tight deadlines and the behavior of prioritizing features over quality aspects.

Rodrigo Rebouças de Almeida, Rafael do Nascimento Ribeiro, Christoph Treude et al.

Incorporating the business perspective into prioritizing technical debt is essential to contribute to decision making in industry. In this paper, we evolve and evaluate a business-driven approach for technical debt prioritization. The approach was evaluated during a five-month industrial case study with business and technical stakeholders' active participation. The results show that the approach contributed to aligning business criteria between the business and technical stakeholders. We also observed a downward trend in the amount of technical debt that affects high-value business assets. Moreover, we identified eight business factors that affect the decision making related to the prioritization of technical debt. The study results suggest that the proposed business-driven technical debt prioritization approach can help teams to focus their efforts on paying off the business' most relevant debt.

Rodrigo Rebouças de Almeida, Christoph Treude, Uirá Kulesza

Technical debt is a pervasive problem in software development. Software development teams have to prioritize debt items and determine whether they should address debt or develop new features at any point in time. This paper presents "Tracy", a framework for the prioritization of technical debt using a business-driven approach built on top of business processes. The current stage of the proposed framework is at the beginning of the third phase of Design Science Research, which is usually divided into the phases of exploration, engineering, and evaluation. The exploration and engineering phases involved the participation of 49 professionals from 12 different groups of three companies. The initial evaluation shows that the presented framework is coherent in its structure and that its results contribute to business-driven decision making on technical debt prioritization.

Rodrigo Rebouças de Almeida, Uirá Kulesza, Christoph Treude et al.

Technical debt (TD) is a metaphor to describe the trade-off between short-term workarounds and long-term goals in software development. Despite being widely used to explain technical issues in business terms, industry and academia still lack a proper way to manage technical debt while explicitly considering business priorities. In this paper, we report on a multiple-case study of how two big software development companies handle technical debt items, and we show how taking the business perspective into account can improve the decision making for the prioritization of technical debt. We also propose a first step toward an approach that uses business process management (BPM) to manage technical debt. We interviewed a set of IT business stakeholders, and we collected and analyzed different sets of technical debt items, comparing how these items would be prioritized using a purely technical versus a business-oriented approach. We found that the use of business process management to support technical debt management makes the technical debt prioritization decision process more aligned with business expectations. We also found evidence that the business process management approach can help technical debt management achieve business objectives.