Beyond the YAML File: Understanding Real-World GitHub Actions Workflow Adoption
For software engineering researchers and practitioners, this work provides empirical insights into real-world CI/CD usage and failure response patterns, though the findings are largely descriptive and hypotheses require future validation.
This study analyzes 258,300 workflow runs from 952 repositories to understand how developers respond to GitHub Actions failures and how workflows are used. It identifies three failure response patterns, finds that higher usage intensity correlates with lower failure rates, and reveals a configuration-usage gap where configured workflows may be disabled or unused.
Continuous Integration and Continuous Deployment (CI/CD) have become fundamental to modern software development, with GitHub Actions (GHA) emerging as a dominant automation platform. In this study, we analyze real-world execution records of GHA, examining how developers react to workflow failures, how these workflows are utilized by projects, and how these aspects relate to project characteristics. We quantitatively analyze 258,300 workflow run records from 952 repositories and perform an in-depth qualitative analysis of 21 selected, diverse GitHub repositories to understand how maintainers and contributors interact with workflow results. We identify three distinct failure response patterns, observe that higher usage intensity of GHA workflows correlates with lower failure rates, and uncover a configuration-usage gap where the presence of configuration files masks disabled or unused workflows. Moreover, our qualitative analysis of relationships between project characteristics and utilization patterns yields five hypotheses for future validation.