Continual Reinforcement Learning for HVAC Systems Control: Integrating Hypernetworks and Transfer Learning
This work addresses energy consumption and operational costs in building management, supporting sustainability goals, but it is incremental as it builds on existing hypernetwork and transfer learning techniques.
The paper tackled the problem of sample inefficiency and limited generalization in reinforcement learning for HVAC systems control by introducing a model-based framework with hypernetworks, achieving rapid convergence within 5 episodes and outperforming model-free methods.
Buildings with Heating, Ventilation, and Air Conditioning (HVAC) systems play a crucial role in ensuring indoor comfort and efficiency. While traditionally governed by physics-based models, the emergence of big data has enabled data-driven methods like Deep Reinforcement Learning (DRL). However, Reinforcement Learning (RL)-based techniques often suffer from sample inefficiency and limited generalization, especially across varying HVAC systems. We introduce a model-based reinforcement learning framework that uses a Hypernetwork to continuously learn environment dynamics across tasks with different action spaces. This enables efficient synthetic rollout generation and improved sample usage. Our approach demonstrates strong backward transfer in a continual learning setting after training on a second task, minimal fine-tuning on the first task allows rapid convergence within just 5 episodes and thus outperforming Model Free Reinforcement Learning (MFRL) and effectively mitigating catastrophic forgetting. These findings have significant implications for reducing energy consumption and operational costs in building management, thus supporting global sustainability goals. Keywords: Deep Reinforcement Learning, HVAC Systems Control, Hypernetworks, Transfer and Continual Learning, Catastrophic Forgetting