Semantic-Aware Scheduling for GPU Clusters with Large Language Models
This addresses inefficiencies in deep learning scheduling for GPU cluster users, offering a non-intrusive enhancement to existing systems.
The paper tackled the problem of GPU cluster schedulers being blind to job semantics, which causes inefficiencies like high profiling overhead and poor failure handling, by proposing SchedMate, a framework that uses LLMs to extract insights from unstructured data sources, resulting in up to a 1.91x reduction in average job completion times.
Deep learning (DL) schedulers are pivotal in optimizing resource allocation in GPU clusters, but operate with a critical limitation: they are largely blind to the semantic context of the jobs they manage. This forces them to rely on limited metadata, leading to high profiling overhead, unreliable duration estimation, inadequate failure handling, and poor observability. To this end, we propose SchedMate, a framework that bridges this semantic gap by systematically extracting deep insights from overlooked, unstructured data sources: source code, runtime logs, and historical jobs. SchedMate enhances existing schedulers non-intrusively through three LLM-based components. Our implementation integrates seamlessly with existing deep learning schedulers. Evaluations on a 128-GPU physical cluster and extensive simulations on production traces show SchedMate reduces average job completion times by up to 1.91x, substantially enhancing the scheduling performance, demonstrating the critical role of semantic-awareness in modern DL scheduling.