Qualities of Quality: A Tertiary Review of Software Quality Measurement Research
This work addresses methodological weaknesses in software quality measurement research for software engineering researchers, but it is incremental as it synthesizes existing reviews rather than introducing new empirical findings.
This paper conducted a tertiary review of software quality measurement research by analyzing 75 secondary studies from 7,811 articles, identifying five conceptual perspectives and three key challenges such as validity issues and misuse of machine learning. It proposes a theoretical framework to address these weaknesses and improve research practices.
This paper presents a tertiary review of software quality measurement research. To conduct this review, we examined an initial dataset of 7,811 articles and found 75 relevant and high-quality secondary analyses of software quality research. Synthesizing this body of work, we offer an overview of perspectives, measurement approaches, and trends. We identify five distinct perspectives that conceptualize quality as heuristic, as maintainability, as a holistic concept, as structural features of software, and as dependability. We also identify three key challenges. First, we find widespread evidence of validity questions with common measures. Second, we observe the application of machine learning methods without adequate evaluation. Third, we observe the use of aging datasets. Finally, from these observations, we sketch a path toward a theoretical framework that will allow software engineering researchers to systematically confront these weaknesses while remaining grounded in the experiences of developers and the real world in which code is ultimately deployed.