Measuring Delivery Consistency in Practice: A DORA Extension from a Multi-Platform Release Setting
For engineering teams using DORA metrics, this work addresses the limitation of first-moment statistics by providing a diagnostic tool to detect irregular deployment cadences, though it is an incremental extension of an existing framework.
The authors propose Delivery Consistency (DC), a bounded second-moment measure of cadence regularity, as an extension to the DORA framework. Applied to 120 weeks of data from a multi-platform release setting, DC distinguished platforms with identical DORA tiers but different cadence regularity and identified shared organizational constraints.
The DevOps Research and Assessment (DORA) framework is the most widely adopted measurement system for performance measurement across engineering teams. However, every DORA metric is a first-moment statistic or a simple ratio, which limits the potential insights into engineering process. For example, metrics like Deployment Frequency do not capture the distributional shape of deployment timing, so teams with identical measures can deploy on a metronomic cadence or in undesirably erratic bursts. We have been developing and piloting Delivery Consistency (DC), a bounded second-moment measure of cadence regularity derived from the coefficient of variation of inter-release intervals. In conjunction with other DORA concepts, we integrated DC into the Delivery Health Matrix, an eight-archetype diagnostic that maps joint readings to differentiated interventions. We report an experience evaluation on a four-platform software delivery group using 120 weeks of data extracted from our Jira, GitHub, and Firebase records. DC allowed us to distinguish platforms with identical DORA tier placements but different cadence regularity, and the Matrix summarized the readings into an archetype that pointed at a shared organization or procedural constraint.