Scalable Deployment of AI Time-series Models for IoT
This addresses the problem of scalable AI model deployment for IoT practitioners, but it appears incremental as it builds on existing cloud and serverless frameworks.
The paper tackles the challenge of managing and deploying large numbers of AI time-series models in IoT applications by introducing IBM Research Castor, a cloud-native system that supports model reuse, automated replication, and parallel execution, achieving scalability for tens of thousands of tasks in real-world smart-grid deployments.
IBM Research Castor, a cloud-native system for managing and deploying large numbers of AI time-series models in IoT applications, is described. Modelling code templates, in Python and R, following a typical machine-learning workflow are supported. A knowledge-based approach to managing model and time-series data allows the use of general semantic concepts for expressing feature engineering tasks. Model templates can be programmatically deployed against specific instances of semantic concepts, thus supporting model reuse and automated replication as the IoT application grows. Deployed models are automatically executed in parallel leveraging a serverless cloud computing framework. The complete history of trained model versions and rolling-horizon predictions is persisted, thus enabling full model lineage and traceability. Results from deployments in real-world smart-grid live forecasting applications are reported. Scalability of executing up to tens of thousands of AI modelling tasks is also evaluated.