Model-Free $μ$ Synthesis via Adversarial Reinforcement Learning
This work provides a model-free approach to μ synthesis, which is an important robust control problem, but it is incremental as it builds on existing methods like DK-iteration and adversarial RL.
The paper tackles the robust control problem of μ synthesis by developing a model-free algorithm that connects adversarial reinforcement learning with μ synthesis, demonstrating its utility through extensive numerical studies.
Motivated by the recent empirical success of policy-based reinforcement learning (RL), there has been a research trend studying the performance of policy-based RL methods on standard control benchmark problems. In this paper, we examine the effectiveness of policy-based RL methods on an important robust control problem, namely $μ$ synthesis. We build a connection between robust adversarial RL and $μ$ synthesis, and develop a model-free version of the well-known $DK$-iteration for solving state-feedback $μ$ synthesis with static $D$-scaling. In the proposed algorithm, the $K$ step mimics the classical central path algorithm via incorporating a recently-developed double-loop adversarial RL method as a subroutine, and the $D$ step is based on model-free finite difference approximation. Extensive numerical study is also presented to demonstrate the utility of our proposed model-free algorithm. Our study sheds new light on the connections between adversarial RL and robust control.