Addressing the Regression Test Problem with Change Impact Analysis for Ada
This addresses the need for safe and efficient regression testing in high-criticality Ada developments, though it is incremental as it adapts existing methods to a specific language.
The paper tackles the regression test selection problem for high-criticality Ada software by introducing a change impact analysis approach that combines static and dynamic data, achieving a 97% reduction in test-case execution on an Ada DNS server example.
The regression test selection problem--selecting a subset of a test-suite given a change--has been studied widely over the past two decades. However, the problem has seen little attention when constrained to high-criticality developments and where a "safe" selection of tests need to be chosen. Further, no practical approaches have been presented for the programming language Ada. In this paper, we introduce an approach to solving the selection problem given a combination of both static and dynamic data for a program and a change-set. We present a change impact analysis for Ada that selects the safe set of tests that need to be re-executed to ensure no regressions. We have implemented the approach in the commercial, unit-testing tool VectorCAST, and validated it on a number of open-source examples. On an example of a fully-functioning Ada implementation of a DNS server (IRONSIDES), the experimental results show a 97% reduction in test-case execution.