New developer metrics: Are comments as crucial as code contributions?
This work addresses the need for better developer evaluation metrics in open-source software platforms, though it appears incremental by extending existing metric approaches.
The study tackled the problem of evaluating developer contributions in open-source projects by proposing new metrics based on commenting and issue-related activity, concluding that commenting can be as valuable as code contributions and that even single-issue contributions indicate strong developer-project relationships, with hit scores affected by dataset sparsity.
Open-source code development has become widespread in recent years. As a result, open-source software platforms have also become popular, and millions of developers from diverse locations are able to contribute to the same projects. On these platforms, various knowledge about them is obtained from user activity. This information is used in the form of developer metrics to solve a variety of challenges. In this study, we proposed new developer metrics, including commenting and issue-related activity, that require less information. We concluded that commenting on any feature of a project can be equally as valuable as code contribution. In addition, besides the quantitative ones, metrics based on only the existence of the activity have been shown to offer also considerable results. We saw that issues were crucial in identifying user contributions. Even if a developer makes a contribution to only one issue on a project, the relation between the developer and the project is tight. The hit scores are relatively lower because of the sparsity problem of our dataset; even so, we believe that we have presented improvable and remarkable new developer metrics.