MLModelCI: An Automatic Cloud Platform for Efficient MLaaS
This provides a tool for multimedia researchers and developers to streamline ML service deployment, though it is incremental as it builds on existing DevOps and containerization techniques.
The authors tackled the problem of manual and tedious deployment of machine learning models as cloud services by developing MLModelCI, an automatic cloud platform that optimizes, profiles, and containerizes models for efficient MLaaS, resulting in a system that bridges the gap between training and serving while being released as open-source.
MLModelCI provides multimedia researchers and developers with a one-stop platform for efficient machine learning (ML) services. The system leverages DevOps techniques to optimize, test, and manage models. It also containerizes and deploys these optimized and validated models as cloud services (MLaaS). In its essence, MLModelCI serves as a housekeeper to help users publish models. The models are first automatically converted to optimized formats for production purpose and then profiled under different settings (e.g., batch size and hardware). The profiling information can be used as guidelines for balancing the trade-off between performance and cost of MLaaS. Finally, the system dockerizes the models for ease of deployment to cloud environments. A key feature of MLModelCI is the implementation of a controller, which allows elastic evaluation which only utilizes idle workers while maintaining online service quality. Our system bridges the gap between current ML training and serving systems and thus free developers from manual and tedious work often associated with service deployment. We release the platform as an open-source project on GitHub under Apache 2.0 license, with the aim that it will facilitate and streamline more large-scale ML applications and research projects.