One evaluation of model-based testing and its automation
This addresses testing efficiency for automotive software, but it is incremental as it compares existing methods in a specific case study.
The study evaluated model-based testing for an automotive network controller, finding that both automatically and manually derived model-based test suites detected significantly more requirements errors than hand-crafted suites from requirements, with a sixfold increase in tests leading to an 11% increase in detected errors.
Model-based testing relies on behavior models for the generation of model traces: input and expected output---test cases---for an implementation. We use the case study of an automotive network controller to assess different test suites in terms of error detection, model coverage, and implementation coverage. Some of these suites were generated automatically with and without models, purely at random, and with dedicated functional test selection criteria. Other suites were derived manually, with and without the model at hand. Both automatically and manually derived model-based test suites detected significantly more requirements errors than hand-crafted test suites that were directly derived from the requirements. The number of detected programming errors did not depend on the use of models. Automatically generated model-based test suites detected as many errors as hand-crafted model-based suites with the same number of tests. A sixfold increase in the number of model-based tests led to an 11% increase in detected errors.