Improving Robustness of Reinforcement Learning for Power System Control with Adversarial Training
This addresses security risks in safety-critical power systems, where RL algorithms are increasingly used but face robustness issues, representing an incremental improvement by applying adversarial training to a specific domain.
The paper tackled the vulnerability of state-of-the-art reinforcement learning agents in power system control to adversarial attacks, demonstrating successful attacks on winning agents from the L2RPN challenge and proposing adversarial training as a defense to improve robustness and avoid infeasible decisions.
Due to the proliferation of renewable energy and its intrinsic intermittency and stochasticity, current power systems face severe operational challenges. Data-driven decision-making algorithms from reinforcement learning (RL) offer a solution towards efficiently operating a clean energy system. Although RL algorithms achieve promising performance compared to model-based control models, there has been limited investigation of RL robustness in safety-critical physical systems. In this work, we first show that several competition-winning, state-of-the-art RL agents proposed for power system control are vulnerable to adversarial attacks. Specifically, we use an adversary Markov Decision Process to learn an attack policy, and demonstrate the potency of our attack by successfully attacking multiple winning agents from the Learning To Run a Power Network (L2RPN) challenge, under both white-box and black-box attack settings. We then propose to use adversarial training to increase the robustness of RL agent against attacks and avoid infeasible operational decisions. To the best of our knowledge, our work is the first to highlight the fragility of grid control RL algorithms, and contribute an effective defense scheme towards improving their robustness and security.