Scalable Multi-Agent Reinforcement Learning for Residential Load Scheduling under Data Governance
This work addresses scalability and privacy issues in real-world residential energy management, representing an incremental improvement over existing distributed MARL approaches.
The paper tackles the challenge of scaling multi-agent reinforcement learning for residential load scheduling under communication constraints and data governance, proposing a distributed actor-critic method that preserves privacy and reduces communication costs while achieving performance comparable to state-of-the-art methods without such constraints.
As a data-driven approach, multi-agent reinforcement learning (MARL) has made remarkable advances in solving cooperative residential load scheduling problems. However, centralized training, the most common paradigm for MARL, limits large-scale deployment in communication-constrained cloud-edge environments. As a remedy, distributed training shows unparalleled advantages in real-world applications but still faces challenge with system scalability, e.g., the high cost of communication overhead during coordinating individual agents, and needs to comply with data governance in terms of privacy. In this work, we propose a novel MARL solution to address these two practical issues. Our proposed approach is based on actor-critic methods, where the global critic is a learned function of individual critics computed solely based on local observations of households. This scheme preserves household privacy completely and significantly reduces communication cost. Simulation experiments demonstrate that the proposed framework achieves comparable performance to the state-of-the-art actor-critic framework without data governance and communication constraints.