Statistical Software for Psychology: Comparing Development Practices Between CRAN and Other Communities
This identifies gaps in software engineering practices for psychology researchers, though it is incremental as it applies existing evaluation methods to a specific domain.
The study measured software development practices in statistical software for psychology, finding that R packages follow good practices while commercial packages are opaque and research projects vary widely.
Different communities rely heavily on software, but use quite different software development practices. {\bf Objective}: We wanted to measure the state of the practice in the area of statistical software for psychology to understand how it compares to best practices. {\bf Method}: We compared and ranked 30 software tools with respect to adherence to best software engineering practices on items that could be measured by end-users. {\bf Results} We found that R packages use quite good practices, that while commercial packages were quite usable, many aspects of their development is too opaque to be measures, and that research projects vary a lot in their practices. {\bf Conclusion} We recommend that more organizations adopt practices similar to those used by CRAN to facilitate success, even for small teams. We also recommend close coupling of source code and documentation, to improve verifiability.