Evaluating Maintainability Prejudices with a Large-Scale Study of Open-Source Projects
This work addresses software engineering decision-making by empirically challenging widely held maintainability beliefs, though it is incremental as it applies existing static analysis metrics to new data.
The study tested 10 common maintainability prejudices using a large-scale analysis of 6,897 GitHub repositories across 5 programming languages, finding that most hypotheses, such as JavaScript having lower maintainability or team-developed code being higher quality, were not supported by the data.
Exaggeration or context changes can render maintainability experience into prejudice. For example, JavaScript is often seen as least elegant language and hence of lowest maintainability. Such prejudice should not guide decisions without prior empirical validation. We formulated 10 hypotheses about maintainability based on prejudices and test them in a large set of open-source projects (6,897 GitHub repositories, 402 million lines, 5 programming languages). We operationalize maintainability with five static analysis metrics. We found that JavaScript code is not worse than other code, Java code shows higher maintainability than C# code and C code has longer methods than other code. The quality of interface documentation is better in Java code than in other code. Code developed by teams is not of higher and large code bases not of lower maintainability. Projects with high maintainability are not more popular or more often forked. Overall, most hypotheses are not supported by open-source data.