Yifu Ding

Yifu Ding, Zixi Chen, Thomas Magnanti

Energy demands from data centers have surged and stressed the grid in recent years. Electric grids require balancing supply and demand every second, motivating demand response (reduction) from large loads, including data centers. This can be achieved by rescheduling jobs on physical machines. Its real-time implementation is uncertain due to fluctuating resource utilization, and rescheduling incurs quality-of-service (QoS) losses that providers are unwilling to disclose. We propose a restless multi-arm bandit (RMAB) framework in which the grid operator requests load reductions without access to detailed job-rescheduling procedures. Using the open-source virtual machine (VM) datasets, we model job arrivals and rescheduling at each data center as a restless arm in a Markov decision process (MDP), and derive Whittle-index-based policies based on the learned transition function via Thompson sampling. To overcome the weakness of an increasingly long learning process due to an enlarged state space, we used a mixed strategy that included a global upper confidence bound (UCB) encoded with trust indices to enhance robustness and accelerate learning. Results show that the proposed mixed-strategy algorithm remains robust across varying state-space sizes and consistently outperforms the pure Thompson-Whittle (TW) algorithm, especially when contextual information is noisy. It also demonstrates superior performance compared to the state-of-the-art EXP4 framework. We provided an open-sourced code for reproducibility.

Yifu Ding, Zixi Chen, Ruicheng Ao et al.

The rapid rise in energy consumption from large-scale AI workloads in data centers placed the increasing pressures on power grids in recent years. Since grids must maintain real-time bal- ance between supply and demand, there has been heightened interest in job flexibility services from data centers. We propose an adaptive, learning-based contextual restless multi-armed bandit (RMAB) framework in which the grid requests load reductions without requiring visibility into job (re)scheduling decisions. With multiple virtual machine (VM) datasets, the core strategy models a cyclic job queue and batch-level (re)scheduling within each data center as a Markov decision process (MDP), and derives Whittle-index-based policies computed from the learned transition and reward functions via Thompson sampling. To address the challenges of enlarging state space and sparse state visits, we introduce domain-knowledge-enriched refined strategies, including adaptive mixed strategies, gated priors, low-rank smoothing, and offline posterior support. Extensive simulations under both baseline and stress scenarios demonstrate that the cellwise best refined variant exceeds TW in every reported cell of the stress sweep at lower computational cost than EXP4. These results demonstrate the economic value of data center flexibility services and underscore the importance of high-quality, open-source AI workload datasets for developing and evaluating adaptive scheduling algorithms.