Inductive-bias-driven Reinforcement Learning For Efficient Schedules in Heterogeneous Clusters
This addresses the challenge of efficient scheduling in heterogeneous clusters for data center operators, offering a more practical alternative to existing methods.
The paper tackles the problem of scheduling workloads onto heterogeneous processors in data centers, presenting Symphony, a framework that uses domain-driven Bayesian reinforcement learning and a sampling-based gradient technique to reduce training data and time, achieving up to 2.2x better performance than black-box approaches.
The problem of scheduling of workloads onto heterogeneous processors (e.g., CPUs, GPUs, FPGAs) is of fundamental importance in modern data centers. Current system schedulers rely on application/system-specific heuristics that have to be built on a case-by-case basis. Recent work has demonstrated ML techniques for automating the heuristic search by using black-box approaches which require significant training data and time, which make them challenging to use in practice. This paper presents Symphony, a scheduling framework that addresses the challenge in two ways: (i) a domain-driven Bayesian reinforcement learning (RL) model for scheduling, which inherently models the resource dependencies identified from the system architecture; and (ii) a sampling-based technique to compute the gradients of a Bayesian model without performing full probabilistic inference. Together, these techniques reduce both the amount of training data and the time required to produce scheduling policies that significantly outperform black-box approaches by up to 2.2x.