Maturity Framework for Enhancing Machine Learning Quality
This addresses the need for rigorous ML quality assessment in business applications, though it is incremental as it builds on existing frameworks.
The paper tackles the problem of ensuring quality, reliability, and reproducibility in machine learning systems by introducing a structured maturity framework for ML governance, validated with empirical evidence from Booking.com to show quality improvement trends and business outcomes.
With the rapid integration of Machine Learning (ML) in business applications and processes, it is crucial to ensure the quality, reliability and reproducibility of such systems. We suggest a methodical approach towards ML system quality assessment and introduce a structured Maturity framework for governance of ML. We emphasize the importance of quality in ML and the need for rigorous assessment, driven by issues in ML governance and gaps in existing frameworks. Our primary contribution is a comprehensive open-sourced quality assessment method, validated with empirical evidence, accompanied by a systematic maturity framework tailored to ML systems. Drawing from applied experience at Booking.com, we discuss challenges and lessons learned during large-scale adoption within organizations. The study presents empirical findings, highlighting quality improvement trends and showcasing business outcomes. The maturity framework for ML systems, aims to become a valuable resource to reshape industry standards and enable a structural approach to improve ML maturity in any organization.