Non-functional Requirements for Machine Learning: Understanding Current Use and Challenges in Industry
This work addresses the problem of adapting requirement engineering practices for ML systems in industry, but it is incremental as it focuses on understanding current use rather than proposing new solutions.
The study investigated how non-functional requirements (NFRs) like performance, transparency, and fairness are handled in machine learning systems in industry, finding examples of their identification, measurement, and associated challenges through interviews with ten practitioners.
Machine Learning (ML) is an application of Artificial Intelligence (AI) that uses big data to produce complex predictions and decision-making systems, which would be challenging to obtain otherwise. To ensure the success of ML-enabled systems, it is essential to be aware of certain qualities of ML solutions (performance, transparency, fairness), known from a Requirement Engineering (RE) perspective as non-functional requirements (NFRs). However, when systems involve ML, NFRs for traditional software may not apply in the same ways; some NFRs may become more prominent or less important; NFRs may be defined over the ML model, data, or the entire system; and NFRs for ML may be measured differently. In this work, we aim to understand the state-of-the-art and challenges of dealing with NFRs for ML in industry. We interviewed ten engineering practitioners working with NFRs and ML. We find examples of (1) the identification and measurement of NFRs for ML, (2) identification of more and less important NFRs for ML, and (3) the challenges associated with NFRs and ML in the industry. This knowledge paints a picture of how ML-related NFRs are treated in practice and helps to guide future RE for ML efforts.