Targeted Test Selection Approach in Continuous Integration
This addresses the problem of slow and resource-intensive testing for software developers in industrial settings, offering a practical incremental improvement over existing methods.
The paper tackles the challenge of efficiently managing testing in continuous integration as codebases grow, proposing Targeted Test Selection (T-TS), a machine learning approach that selects only 15% of tests, reduces execution time by 5.9x, accelerates the pipeline by 5.6x, and detects over 95% of test failures on live industrial data.
In modern software development change-based testing plays a crucial role. However, as codebases expand and test suites grow, efficiently managing the testing process becomes increasingly challenging, especially given the high frequency of daily code commits. We propose Targeted Test Selection (T-TS), a machine learning approach for industrial test selection. Our key innovation is a data representation that represent commits as Bags-of-Words of changed files, incorporates cross-file and additional predictive features, and notably avoids the use of coverage maps. Deployed in production, T-TS was comprehensively evaluated against industry standards and recent methods using both internal and public datasets, measuring time efficiency and fault detection. On live industrial data, T-TS selects only 15% of tests, reduces execution time by $5.9\times$, accelerates the pipeline by $5.6\times$, and detects over 95% of test failures. The implementation is publicly available to support further research and practical adoption.