Heterogeneous Multi-Agent Proximal Policy Optimization for Power Distribution System Restoration
This provides a scalable, constraint-aware solution for power system restoration, addressing practical heterogeneity in microgrids, though it appears incremental as an adaptation of existing RL methods to this domain.
The paper tackles the problem of restoring power distribution systems after large-scale outages by applying a Heterogeneous-Agent Reinforcement Learning framework with Proximal Policy Optimization (HAPPO) to coordinate interconnected microgrids. Results show HAPPO achieves faster convergence and higher restored power than seven baseline methods on IEEE 123-bus and 8500-node systems.
Restoring power distribution systems (PDS) after large-scale outages requires sequential switching operations that reconfigure feeder topology and coordinate distributed energy resources (DERs) under nonlinear constraints such as power balance, voltage limits, and thermal ratings. These challenges make conventional optimization and value-based RL approaches computationally inefficient and difficult to scale. This paper applies a Heterogeneous-Agent Reinforcement Learning (HARL) framework, instantiated through Heterogeneous-Agent Proximal Policy Optimization (HAPPO), to enable coordinated restoration across interconnected microgrids. Each agent controls a distinct microgrid with different loads, DER capacities, and switch counts, introducing practical structural heterogeneity. Decentralized actor policies are trained with a centralized critic to compute advantage values for stable on-policy updates. A physics-informed OpenDSS environment provides full power flow feedback and enforces operational limits via differentiable penalty signals rather than invalid action masking. The total DER generation is capped at 2400 kW, and each microgrid must satisfy local supply-demand feasibility. Experiments on the IEEE 123-bus and IEEE 8500-node systems show that HAPPO achieves faster convergence, higher restored power, and smoother multi-seed training than DQN, PPO, MAES, MAGDPG, MADQN, Mean-Field RL, and QMIX. Results demonstrate that incorporating microgrid-level heterogeneity within the HARL framework yields a scalable, stable, and constraint-aware solution for complex PDS restoration.