A Relearning Approach to Reinforcement Learning for Control of Smart Buildings
This work addresses energy efficiency and comfort in smart buildings, but it is incremental as it builds on existing RL methods for a specific domain.
The paper tackled the problem of non-stationary processes in smart building HVAC control by developing an incremental deep reinforcement learning approach for continual relearning, resulting in improved energy consumption reduction while maintaining comfort compared to a static controller.
This paper demonstrates that continual relearning of control policies using incremental deep reinforcement learning (RL) can improve policy learning for non-stationary processes. We demonstrate this approach for a data-driven 'smart building environment' that we use as a test-bed for developing HVAC controllers for reducing energy consumption of large buildings on our university campus. The non-stationarity in building operations and weather patterns makes it imperative to develop control strategies that are adaptive to changing conditions. On-policy RL algorithms, such as Proximal Policy Optimization (PPO) represent an approach for addressing this non-stationarity, but exploration on the actual system is not an option for safety-critical systems. As an alternative, we develop an incremental RL technique that simultaneously reduces building energy consumption without sacrificing overall comfort. We compare the performance of our incremental RL controller to that of a static RL controller that does not implement the relearning function. The performance of the static controller diminishes significantly over time, but the relearning controller adjusts to changing conditions while ensuring comfort and optimal energy performance.