Intrinsic Properties of Complete Test Suites
This work addresses a theoretical limitation in software testing for model-based systems, but it is incremental as it extends existing completeness concepts to handle blocking test cases.
The paper tackles the problem of completeness in test suites for model-based testing, particularly when test cases can be blocking, and defines a new notion of completeness in this scenario while establishing an upper bound on the number of states for which completeness is possible.
Completeness is a desirable property of test suites. Roughly, completeness guarantees that a non-equivalent implementation under test will always be identified. Several approaches proposed sufficient, and sometimes also necessary, conditions on the specification model and on the test suite in order to guarantee completeness. Usually, these approaches impose several restrictions on the specification and on the implementations, such as requiring them to be reduced or complete. Further, test cases are required to be non-blocking --- that is, they must run to completion --- on both the specification and the implementation models. In this work we deal test cases that can be blocking, we define a new notion that captures completeness, and we characterize test suite completeness in this new scenario. We establish an upper bound on the number of states of implementations beyond which no test suite can be complete, both in the classical sense and in the new scenario with blocking test cases.