Renewable Energy Prediction using Weather Forecasts for Optimal Scheduling in HPC Systems
This addresses energy efficiency and cost reduction for HPC data centers by optimizing renewable energy utilization, but it is incremental as it builds on existing statistical and machine learning methods.
The paper tackles the problem of predicting wind energy availability for scheduling HPC jobs to maximize green energy use, by developing a statistical model from historical weather data and using weather forecasts to precompute schedules, with on-the-fly adjustments based on live data and machine learning.
The objective of the GreenPAD project is to use green energy (wind, solar and biomass) for powering data-centers that are used to run HPC jobs. As a part of this it is important to predict the Renewable (Wind) energy for efficient scheduling (executing jobs that require higher energy when there is more green energy available and vice-versa). For predicting the wind energy we first analyze the historical data to find a statistical model that gives relation between wind energy and weather attributes. Then we use this model based on the weather forecast data to predict the green energy availability in the future. Using the green energy prediction obtained from the statistical model we are able to precompute job schedules for maximizing the green energy utilization in the future. We propose a model which uses live weather data in addition to machine learning techniques (which can predict future deviations in weather conditions based on current deviations from the forecast) to make on-the-fly changes to the precomputed schedule (based on green energy prediction). For this we first analyze the data using histograms and simple statistical tools such as correlation. In addition we build (correlation) regression model for finding the relation between wind energy availability and weather attributes (temperature, cloud cover, air pressure, wind speed / direction, precipitation and sunshine). We also analyze different algorithms and machine learning techniques for optimizing the job schedules for maximizing the green energy utilization.