Scalable and Cost-Efficient ML Inference: Parallel Batch Processing with Serverless Functions
This addresses scalability and cost-efficiency problems for data-intensive applications in limited-resource environments, representing an incremental improvement by applying serverless computing to ML inference.
The paper tackled scalability and cost challenges in ML inference by using serverless functions for parallel batch processing, demonstrating a reduction in execution time by over 95% compared to monolithic approaches at the same cost in a sentiment analysis case study.
As data-intensive applications grow, batch processing in limited-resource environments faces scalability and resource management challenges. Serverless computing offers a flexible alternative, enabling dynamic resource allocation and automatic scaling. This paper explores how serverless architectures can make large-scale ML inference tasks faster and cost-effective by decomposing monolithic processes into parallel functions. Through a case study on sentiment analysis using the DistilBERT model and the IMDb dataset, we demonstrate that serverless parallel processing can reduce execution time by over 95% compared to monolithic approaches, at the same cost.