Data-Driven Learning and Load Ensemble Control
This work addresses grid support services for energy systems by improving control of distributed loads, but it is incremental as it applies an existing learning method to a specific domain.
The paper tackled the problem of controlling aggregated thermostatically controllable loads (TCLs) for demand response programs by applying a model-free reinforcement learning technique called Z-learning to a linearly solvable Markov decision process model, demonstrating efficiency through simulations on HVAC units in a residential testbed.
Demand response (DR) programs aim to engage distributed small-scale flexible loads, such as thermostatically controllable loads (TCLs), to provide various grid support services. Linearly Solvable Markov Decision Process (LS-MDP), a variant of the traditional MDP, is used to model aggregated TCLs. Then, a model-free reinforcement learning technique called Z-learning is applied to learn the value function and derive the optimal policy for the DR aggregator to control TCLs. The learning process is robust against uncertainty that arises from estimating the passive dynamics of the aggregated TCLs. The efficiency of this data-driven learning is demonstrated through simulations on Heating, Cooling & Ventilation (HVAC) units in a testbed neighborhood of residential houses.