Ticket Coverage: Putting Test Coverage into Context
This addresses a major issue for software developers and testers by providing a specific metric to prevent untested code from reaching production, though it is incremental as it builds on existing test coverage concepts.
The paper tackles the problem of untested code changes in software development by introducing ticket coverage, a metric that measures the ratio of changed methods covered by tests per ticket, and shows in an empirical study on a 650k-line Java system that it reveals test gaps and improves transparency.
There is no metric that determines how well the implementation of a ticket has been tested. As a consequence, code changed within the context of a ticket might unintentionally remain untested and get into production. This is a major problem, because changed code is more fault-prone than unchanged code. In this paper, we introduce the metric ticket coverage which puts test coverage into the context of tickets. For each ticket, it determines the ratio of changed methods covered by automated or manual tests. We conducted an empirical study on an industrial system consisting of 650k lines of Java code and show that ticket coverage brings transparency into the test state of tickets and reveals relevant test gaps.