A new perspective on the competent programmer hypothesis through the reproduction of bugs with repeated mutations
This work addresses a foundational assumption in mutation testing for software engineering, but it is incremental as it builds on existing contradictory claims.
The study investigates the competent programmer hypothesis by attempting to recreate real-world bugs through mutation chains, finding that while the hypothesis appears valid, mutation testing lacks operators to generate representative bugs.
The competent programmer hypothesis states that most programmers are competent enough to create correct or almost correct source code. Because this implies that bugs should usually manifest through small variations of the correct code, the competent programmer hypothesis is one of the fundamental assumptions of mutation testing. Unfortunately, it is still unclear if the competent programmer hypothesis holds and past research presents contradictory claims. Within this article, we provide a new perspective on the competent programmer hypothesis and its relation to mutation testing. We try to re-create real-world bugs through chains of mutations to understand if there is a direct link between mutation testing and bugs. The lengths of these paths help us to understand if the source code is really almost correct, or if large variations are required. Our results indicate that while the competent programmer hypothesis seems to be true, mutation testing is missing important operators to generate representative real-world bugs.