A Meta-Summary of Challenges in Building Products with ML Components -- Collecting Experiences from 4758+ Practitioners
This work aggregates practitioner experiences to guide research priorities in software engineering for ML, but it is incremental as it summarizes existing studies rather than introducing new solutions.
The paper conducted a meta-summary study by analyzing 50 relevant papers that interacted with over 4758 practitioners to identify common challenges in building products with machine learning components, highlighting over 500 mentions of challenges to help prioritize research and education.
Incorporating machine learning (ML) components into software products raises new software-engineering challenges and exacerbates existing challenges. Many researchers have invested significant effort in understanding the challenges of industry practitioners working on building products with ML components, through interviews and surveys with practitioners. With the intention to aggregate and present their collective findings, we conduct a meta-summary study: We collect 50 relevant papers that together interacted with over 4758 practitioners using guidelines for systematic literature reviews. We then collected, grouped, and organized the over 500 mentions of challenges within those papers. We highlight the most commonly reported challenges and hope this meta-summary will be a useful resource for the research community to prioritize research and education in this field.