Stability-certified reinforcement learning: A control-theoretic perspective
This addresses the critical issue of ensuring stability in RL-based controllers for safety-critical applications like autonomous systems and power grids, though it appears incremental by building on control-theoretic methods.
The paper tackles the problem of certifying stability for reinforcement learning policies when connected to nonlinear dynamical systems, showing that regulating input-output gradients enables robust stability guarantees through semidefinite programming, with empirical results demonstrating high performance and stable learning in decentralized control tasks like multi-flight formation and power system frequency regulation.
We investigate the important problem of certifying stability of reinforcement learning policies when interconnected with nonlinear dynamical systems. We show that by regulating the input-output gradients of policies, strong guarantees of robust stability can be obtained based on a proposed semidefinite programming feasibility problem. The method is able to certify a large set of stabilizing controllers by exploiting problem-specific structures; furthermore, we analyze and establish its (non)conservatism. Empirical evaluations on two decentralized control tasks, namely multi-flight formation and power system frequency regulation, demonstrate that the reinforcement learning agents can have high performance within the stability-certified parameter space, and also exhibit stable learning behaviors in the long run.