Some SonarQube Issues have a Significant but SmallEffect on Faults and Changes. A large-scale empirical study
This provides incremental insights for software practitioners and researchers on prioritizing TD refactoring and improving tool accuracy.
The study analyzed the impact of Technical Debt (TD) items, detected by SonarQube, on code changes and fault-proneness in 33 Java projects, finding that clean classes are slightly less change-prone than dirty ones but with no difference in fault-proneness, and identified inconsistencies in SonarQube's severity assignments.
Context. Companies commonly invest effort to remove technical issues believed to impact software qualities, such as removing anti-patterns or coding styles violations. Objective. Our aim is to analyze the diffuseness of Technical Debt (TD) items in software systems and to assess their impact on code changes and fault-proneness, considering also the type of TD items and their severity. Method. We conducted a case study among 33 Java projects from the Apache Software Foundation (ASF) repository. We analyzed 726 commits containing 27K faults and 12M changes. The projects violated 173 SonarQube rules generating more than 95K TD items in more than 200K classes. Results. Clean classes (classes not affected by TD items) are less change-prone than dirty ones, but the difference between the groups is small. Clean classes are slightly more change-prone than classes affected by TD items of type Code Smell or Security Vulnerability. As for fault-proneness, there is no difference between clean and dirty classes. Moreover, we found a lot of incongruities in the type and severity level assigned by SonarQube. Conclusions. Our result can be useful for practitioners to understand which TD items they should refactor and for researchers to bridge the missing gaps. They can also support companies and tool vendors in identifying TD items as accurately as possible.