React to Surprises: Stable-by-Design Neural Feedback Control and the Youla-REN
This work addresses the challenge of designing stable neural feedback controllers for nonlinear systems with partial observations, which is crucial for safe and reliable autonomous systems, though it appears incremental in combining existing concepts.
The authors tackled the problem of ensuring closed-loop stability in learning-based control by proposing a parameterization of stabilizing nonlinear policies using a nonlinear Youla-Kucera parameterization combined with robust neural networks, which guarantees stability by construction and allows unconstrained optimization. They demonstrated its utility in numerical experiments for learning controllers with built-in stability certificates in scenarios like economic rewards and uncertain systems.
We study parameterizations of stabilizing nonlinear policies for learning-based control. We propose a structure based on a nonlinear version of the Youla-Kucera parameterization combined with robust neural networks such as the recurrent equilibrium network (REN). The resulting parameterizations are unconstrained, and hence can be searched over with first-order optimization methods, while always ensuring closed-loop stability by construction. We study the combination of (a) nonlinear dynamics, (b) partial observation, and (c) incremental closed-loop stability requirements (contraction and Lipschitzness). We find that with any two of these three difficulties, a contracting and Lipschitz Youla parameter always leads to contracting and Lipschitz closed loops. However, if all three hold, then incremental stability can be lost with exogenous disturbances. Instead, a weaker condition is maintained, which we call d-tube contraction and Lipschitzness. We further obtain converse results showing that the proposed parameterization covers all contracting and Lipschitz closed loops for certain classes of nonlinear systems. Numerical experiments illustrate the utility of our parameterization when learning controllers with built-in stability certificates for: (i) "economic" rewards without stabilizing effects; (ii) short training horizons; and (iii) uncertain systems.