Industry Practitioners Perspectives on AI Model Quality: Perceptions, Challenges, and Solutions
This work addresses the problem of aligning AI research with industry needs by identifying practitioner priorities, though it is incremental as it synthesizes existing perceptions rather than proposing new methods.
The study investigated AI model quality attributes in industry by interviewing 15 practitioners and surveying 50, finding that priorities like efficiency over correctness vary by context, with data imbalance as a key challenge and active learning as a common solution.
Artificial Intelligence (AI) is now used across nearly every industry, making AI model quality essential for building reliable and trustworthy systems. Historically, correctness has been the main focus, but industry AI models must also satisfy many other important quality attributes. To understand how these attributes are perceived, the challenges they create, and the solutions used in practice, we identify nine key quality attributes and interview 15 AI practitioners from diverse backgrounds. The interviews show that practitioners prioritize attributes differently depending on context. For example, efficiency can matter more than correctness in real-time applications, while scalability and deployability are no longer seen as primary concerns. Data imbalance emerges as a major obstacle to maintaining model correctness and robustness, and practitioners commonly use mitigation strategies such as active learning. We validate our main findings with a survey of 50 practitioners, which shows that most of the findings are widely recognized. These results can help researchers focus on the attributes practitioners value most and avoid improving one attribute at the expense of others that are considered more critical.