DRAS-CQSim: A Reinforcement Learning based Framework for HPC Cluster Scheduling
This addresses the problem of time-consuming and error-prone manual scheduling for HPC system administrators, though it appears incremental as it builds on existing reinforcement learning methods.
The authors tackled the challenge of manually designing scheduling policies for complex HPC systems by developing DRAS-CQSim, a reinforcement learning framework that automatically learns optimal policies, enabling system administrators to quickly obtain customized solutions.
For decades, system administrators have been striving to design and tune cluster scheduling policies to improve the performance of high performance computing (HPC) systems. However, the increasingly complex HPC systems combined with highly diverse workloads make such manual process challenging, time-consuming, and error-prone. We present a reinforcement learning based HPC scheduling framework named DRAS-CQSim to automatically learn optimal scheduling policy. DRAS-CQSim encapsulates simulation environments, agents, hyperparameter tuning options, and different reinforcement learning algorithms, which allows the system administrators to quickly obtain customized scheduling policies.