Assessing Developer Beliefs: A Reply to "Perceptions, Expectations, and Challenges in Defect Prediction"
It addresses the need for validating qualitative insights with data in software engineering, though it is incremental as it builds directly on a prior study.
This paper tackles the problem of discrepancies between developer beliefs and empirical evidence in defect prediction, by quantitatively analyzing prior qualitative findings to identify and report these mismatches, aiming to prevent wasted effort and overlooked issues.
It can be insightful to extend qualitative studies with a secondary quantitative analysis (where the former suggests insightful questions that the latter can answer). Documenting developer beliefs should be the start, not the end, of Software Engineering research. Once prevalent beliefs are found, they should be checked against real-world data. For example, this paper finds several notable discrepancies between empirical evidence and the developer beliefs documented in Wan et al.'s recent TSE paper "Perceptions, expectations, and challenges in defect prediction". By reporting these discrepancies we can stop developers (a) wasting time on inconsequential matters or (b) ignoring important effects. For the future, we would encourage more "extension studies" of prior qualitative results with quantitative empirical evidence.