A Convex Parameterization of Robust Recurrent Neural Networks
This provides robust RNNs with formal guarantees for applications requiring stability, such as system identification, though it is incremental in extending existing convex parameterization methods.
The paper tackles the lack of stability and robustness guarantees in recurrent neural networks (RNNs) by formulating convex sets of RNNs with global exponential stability and bounds on incremental ℓ₂ gain, proving that this model structure includes all previously-proposed convex sets of stable RNNs and all stable linear dynamical systems.
Recurrent neural networks (RNNs) are a class of nonlinear dynamical systems often used to model sequence-to-sequence maps. RNNs have excellent expressive power but lack the stability or robustness guarantees that are necessary for many applications. In this paper, we formulate convex sets of RNNs with stability and robustness guarantees. The guarantees are derived using incremental quadratic constraints and can ensure global exponential stability of all solutions, and bounds on incremental $ \ell_2 $ gain (the Lipschitz constant of the learned sequence-to-sequence mapping). Using an implicit model structure, we construct a parametrization of RNNs that is jointly convex in the model parameters and stability certificate. We prove that this model structure includes all previously-proposed convex sets of stable RNNs as special cases, and also includes all stable linear dynamical systems. We illustrate the utility of the proposed model class in the context of non-linear system identification.