Landscaping Systematic Mapping Studies in Software Engineering: A Tertiary Study
This tertiary study identifies gaps and quality issues in systematic mapping studies for software engineering researchers, but it is incremental as it builds on existing secondary studies without introducing new methods.
The study synthesized 210 Systematic Mapping Studies (SMSs) in Software Engineering to analyze trends and quality, finding that software design and construction are the most active areas, while some areas lack coverage, and SMS quality is improving but often low due to missing information.
Context: A number of Systematic Mapping Studies (SMSs) that cover Software Engineering (SE) are reported in literature. Tertiary studies synthesize the secondary studies to provide a holistic view of an area. Objectives: We synthesize SMSs in SE to provide insights into existing SE areas and to investigate the trends and quality of SMSs. Methodology: We use Systematic Literature Review protocol to analyze and map the SMSs in SE, till August 2017, to SE Body of Knowledge (SWEBOK). Results: We analyze 210 SMSs and results show that: (1) Software design and construction are most active areas in SE; (2) Some areas lack SMSs, including mathematical foundations, software configuration management, and SE tools; (3) The quality of SMSs is improving with time; (4) SMSs in journals have higher quality than SMSs in conferences and are cited more often; (5) Low quality in SMSs can be attributed to a lack of quality assessment in SMSs and not reporting information about the primary studies. Conclusion: There is a potential for more SMSs in some SE areas. A number of SMSs do not provide the required information for an SMS, which leads to a low quality score.