Reinforcement Learning for Machine Learning Model Deployment: Evaluating Multi-Armed Bandits in ML Ops Environments
This work addresses the need for more adaptive and automated model deployment in ML Ops environments, reducing manual intervention and mitigating post-deployment risks, though it is incremental as it builds on existing RL methods.
The paper tackled the problem of static model deployment in ML Ops by investigating reinforcement learning, specifically multi-armed bandit algorithms, to dynamically manage decisions, finding that RL-based approaches match or exceed traditional methods in performance across two real-world datasets.
In modern ML Ops environments, model deployment is a critical process that traditionally relies on static heuristics such as validation error comparisons and A/B testing. However, these methods require human intervention to adapt to real-world deployment challenges, such as model drift or unexpected performance degradation. We investigate whether reinforcement learning, specifically multi-armed bandit (MAB) algorithms, can dynamically manage model deployment decisions more effectively. Our approach enables more adaptive production environments by continuously evaluating deployed models and rolling back underperforming ones in real-time. We test six model selection strategies across two real-world datasets and find that RL based approaches match or exceed traditional methods in performance. Our findings suggest that reinforcement learning (RL)-based model management can improve automation, reduce reliance on manual interventions, and mitigate risks associated with post-deployment model failures.