Complementing Model Learning with Mutation-Based Fuzzing
This work addresses the problem of improving counterexample generation for model learning in reactive software systems, showing complementary benefits but is incremental as it builds on existing testing and fuzzing methods.
The paper tackled the challenge of efficiently obtaining counterexamples in learning algorithms for finite state machine models, comparing and combining conformance testing and mutation-based fuzzing on RERS challenge problems. For LTL problems, the fuzzer confirmed learning success with no additional counterexamples, while for reachability problems, the fuzzer discovered more reachable error states than the learner and tester, though each method found some unique states.
An ongoing challenge for learning algorithms formulated in the Minimally Adequate Teacher framework is to efficiently obtain counterexamples. In this paper we compare and combine conformance testing and mutation-based fuzzing methods for obtaining counterexamples when learning finite state machine models for the reactive software systems of the Rigorous Exampination of Reactive Systems (RERS) challenge. We have found that for the LTL problems of the challenge the fuzzer provided an independent confirmation that the learning process had been successful, since no additional counterexamples were found. For the reachability problems of the challenge, however, the fuzzer discovered more reachable error states than the learner and tester, albeit in some cases the learner and tester found some that were not discovered by the fuzzer. This leads us to believe that these orthogonal approaches are complementary in the context of model learning.