Centrally Coordinated Multi-Agent Reinforcement Learning for Power Grid Topology Control
This study addresses the problem of efficient power grid operation for power grid operators and researchers, providing an incremental solution to the existing challenges in the field.
The study tackled the challenge of power grid operation complexity by proposing a centrally coordinated multi-agent architecture, resulting in higher sample efficiency and superior final performance compared to baseline approaches. The proposed architecture showed high potential for application in higher-dimensional power grid settings.
Power grid operation is becoming more complex due to the increase in generation of renewable energy. The recent series of Learning To Run a Power Network (L2RPN) competitions have encouraged the use of artificial agents to assist human dispatchers in operating power grids. However, the combinatorial nature of the action space poses a challenge to both conventional optimizers and learned controllers. Action space factorization, which breaks down decision-making into smaller sub-tasks, is one approach to tackle the curse of dimensionality. In this study, we propose a centrally coordinated multi-agent (CCMA) architecture for action space factorization. In this approach, regional agents propose actions and subsequently a coordinating agent selects the final action. We investigate several implementations of the CCMA architecture, and benchmark in different experimental settings against various L2RPN baseline approaches. The CCMA architecture exhibits higher sample efficiency and superior final performance than the baseline approaches. The results suggest high potential of the CCMA approach for further application in higher-dimensional L2RPN as well as real-world power grid settings.