PowerGridworld: A Framework for Multi-Agent Reinforcement Learning in Power Systems
This provides a practical tool for researchers and engineers in power systems to develop and test multi-agent RL policies, though it is incremental as it builds on existing MARL frameworks.
The authors tackled the lack of a lightweight, modular framework for creating multi-agent reinforcement learning environments in power systems, resulting in the open-source PowerGridworld package that enables rapid prototyping and integration with existing training tools.
We present the PowerGridworld software package to provide users with a lightweight, modular, and customizable framework for creating power-systems-focused, multi-agent Gym environments that readily integrate with existing training frameworks for reinforcement learning (RL). Although many frameworks exist for training multi-agent RL (MARL) policies, none can rapidly prototype and develop the environments themselves, especially in the context of heterogeneous (composite, multi-device) power systems where power flow solutions are required to define grid-level variables and costs. PowerGridworld is an open-source software package that helps to fill this gap. To highlight PowerGridworld's key features, we present two case studies and demonstrate learning MARL policies using both OpenAI's multi-agent deep deterministic policy gradient (MADDPG) and RLLib's proximal policy optimization (PPO) algorithms. In both cases, at least some subset of agents incorporates elements of the power flow solution at each time step as part of their reward (negative cost) structures.