Joint Scheduling of Deferrable and Nondeferrable Demand with Colocated Stochastic Supply
For energy system operators, this work provides a tractable scheduling framework for integrating stochastic renewable supply with flexible and inflexible demands under time-varying pricing.
This paper addresses the scheduling of deferrable and nondeferrable electric demands with colocated stochastic supply and grid electricity. It introduces the Principle of Procrastination, reducing the policy space to finite dimensions, and proposes a reinforcement learning algorithm that closely approximates the optimal policy, outperforming standard benchmarks in numerical experiments.
We investigate the problem of serving deferrable and nondeferrable electric demands with colocated stochastic supply and grid-imported electricity. Deferrable demands arrive randomly and can be delayed within their service deadlines. Nondeferrable demands are always present and must be served immediately, but the quantity served depends on the cost of electricity. Colocated supply is stochastic with zero marginal cost. It can be used to meet demand or exported to the grid to maximize profit. The stochasticity of demands and local supply makes optimal scheduling a Markov decision process with continuous (uncountable) state and action spaces. Under deterministic, time-varying, and piecewise-linear retail pricing of electricity, we show that the optimal demand scheduling follows the {\em Principle of Procrastination}, which reduces the infinite-dimensional policy space to a finite-dimensional Euclidean space defined by three procrastination parameters for each deferrable demand. For settings in which the underlying probability distributions are unknown, we propose a {\em Procrastination Threshold Reinforcement Learning} algorithm. Numerical experiments based on real-world test data confirm that the proposed threshold learning algorithm closely approximates the optimal policy and outperforms standard benchmarks.