"Two-Stagification": Job Dispatching in Large-Scale Clusters via a Two-Stage Architecture
For operators of large-scale computing clusters, this work offers a simpler architectural alternative to complex dispatching policies that can significantly improve performance.
The paper introduces a two-stage cluster architecture that uses classical dispatching policies with a service-time threshold to separate large and short jobs, achieving mean response times close to advanced size- and state-aware methods in large-scale clusters.
A continuing effort is devoted to devising effective dispatching policies for clusters of First Come First Served servers. Although the optimal solution for dispatchers aware of both job size and server state remains elusive, lower bounds and strong heuristics are known. In this paper, we introduce a two-stage cluster architecture that applies classical Round Robin, Join Idle Queue, and Least Work Left dispatching schemes, coupled with an optimized service-time threshold to separate large jobs from shorter ones. Using both synthetic (Weibull) workloads and real Google data center traces, we demonstrate that our two-stage approach greatly improves upon the corresponding single-stage policies and closely approaches the performance of advanced size- and state-aware methods. Our results highlight that careful architectural design-rather than increased complexity at the dispatcher-can yield significantly better mean response times in large-scale computing environments.