Complete Requirements-based Testing with Finite State Machines
This work addresses testing challenges for safety-critical systems, offering incremental improvements in test generation methods.
The paper tackles the problem of requirements-based testing for deterministic finite state machines by introducing two test generation strategies with proven fault coverage guarantees, where the first strategy is exhaustive and the second is complete, with complexity considerations favoring the first for practical use.
In this paper, new contributions to requirements-based testing with deterministic finite state machines are presented. Elementary requirements are specified as triples consisting of a state in the reference model, an input, and the expected reaction of the system under test defined by a set of admissible outputs, allowing for different implementation variants. Composite requirements are specified as collections of elementary ones. Two requirements-driven test generation strategies are introduced, and their fault coverage guarantees are proven. The first is exhaustive in the sense that it produces test suites guaranteeing requirements satisfaction if the test suite is passed. If the test suite execution fails for a given implementation, however, this does not imply that the requirement has been violated. Instead, the failure may indicate an arbitrary violation of I/O-equivalence, which could be unrelated to the requirement under test. The second strategy is complete in the sense that it produces test suites guaranteeing requirements satisfaction if and only if the suite is passed. Complexity considerations indicate that for practical application, the first strategy should be preferred to the second. Typical application scenarios for this approach are safety-critical systems, where safety requirements should be tested with maximal thoroughness, while user requirements might be checked with lesser effort, using conventional testing heuristics.