GOGH: Correlation-Guided Orchestration of GPUs in Heterogeneous Clusters
This addresses the challenge of sustainable resource management in mixed-generation GPU clusters for organizations unable to upgrade hardware, though it appears incremental as it builds on existing learning-based allocation methods.
The paper tackles the problem of efficient resource allocation for machine learning workloads in heterogeneous hardware clusters, proposing a learning-based architecture that uses neural networks to estimate and optimize resource usage, resulting in an adaptive system that minimizes energy consumption while meeting performance requirements.
The growing demand for computational resources in machine learning has made efficient resource allocation a critical challenge, especially in heterogeneous hardware clusters where devices vary in capability, age, and energy efficiency. Upgrading to the latest hardware is often infeasible, making sustainable use of existing, mixed-generation resources essential. In this paper, we propose a learning-based architecture for managing machine learning workloads in heterogeneous clusters. The system operates online, allocating resources to incoming training or inference requests while minimizing energy consumption and meeting performance requirements. It uses two neural networks: the first provides initial estimates of how well a new model will utilize different hardware types and how it will affect co-located models. An optimizer then allocates resources based on these estimates. After deployment, the system monitors real performance and uses this data to refine its predictions via a second neural network. This updated model improves estimates not only for the current hardware but also for hardware not initially allocated and for co-location scenarios not yet observed. The result is an adaptive, iterative approach that learns over time to make more effective resource allocation decisions in heterogeneous deep learning clusters.