BEAVER: Building Environments with Assessable Variation for Evaluating Multi-Objective Reinforcement Learning
This work addresses the scalability and generalization challenges of RL in building control, which is incremental as it formulates and benchmarks the problem without introducing a new method.
The paper tackled the problem of generalizing reinforcement learning for building energy management across varied operational contexts, showing that existing multi-objective RL methods achieve reasonable trade-offs but degrade under certain environment variations.
Recent years have seen significant advancements in designing reinforcement learning (RL)-based agents for building energy management. While individual success is observed in simulated or controlled environments, the scalability of RL approaches in terms of efficiency and generalization across building dynamics and operational scenarios remains an open question. In this work, we formally characterize the generalization space for the cross-environment, multi-objective building energy management task, and formulate the multi-objective contextual RL problem. Such a formulation helps understand the challenges of transferring learned policies across varied operational contexts such as climate and heat convection dynamics under multiple control objectives such as comfort level and energy consumption. We provide a principled way to parameterize such contextual information in realistic building RL environments, and construct a novel benchmark to facilitate the evaluation of generalizable RL algorithms in practical building control tasks. Our results show that existing multi-objective RL methods are capable of achieving reasonable trade-offs between conflicting objectives. However, their performance degrades under certain environment variations, underscoring the importance of incorporating dynamics-dependent contextual information into the policy learning process.