Reinforcement Learning for Test Case Prioritization
This work addresses the problem of efficiently prioritizing test cases for large software development teams to maintain software quality without compromising CI pipeline speed, which is an incremental improvement for software engineers.
This paper explores the application of Reinforcement Learning (RL) to optimize test case prioritization in Continuous Integration (CI) environments, specifically addressing the challenge of exhaustive testing in large software projects. The study demonstrates that an RL approach using a Network Approximator and Test Case Failure Reward achieved a Normalized percentage of Fault Detection (NAPFD) of over 0.6 on novel data from a financial institution.
In modern software engineering, Continuous Integration (CI) has become an indispensable step towards systematically managing the life cycles of software development. Large companies struggle with keeping the pipeline updated and operational, in useful time, due to the large amount of changes and addition of features, that build on top of each other and have several developers, working on different platforms. Associated with such software changes, there is always a strong component of Testing. As teams and projects grow, exhaustive testing quickly becomes inhibitive, becoming adamant to select the most relevant test cases earlier, without compromising software quality. This paper extends recent studies on applying Reinforcement Learning to optimize testing strategies. We test its ability to adapt to new environments, by testing it on novel data extracted from a financial institution, yielding a Normalized percentage of Fault Detection (NAPFD) of over $0.6$ using the Network Approximator and Test Case Failure Reward. Additionally, we studied the impact of using Decision Tree (DT) Approximator as a model for memory representation, which failed to produce significant improvements relative to Artificial Neural Networks.