Stability and Performance Analysis of Discrete-Time ReLU Recurrent Neural Networks
This work addresses stability analysis for ReLU RNNs, which is an incremental improvement for researchers in control theory and neural network verification.
The paper tackled the problem of analyzing stability and performance for discrete-time ReLU recurrent neural networks by deriving sufficient conditions using Lyapunov/dissipativity theory and Quadratic Constraints, showing that positive homogeneity does not expand the QC class and demonstrating the effect of lifting horizon in examples.
This paper presents sufficient conditions for the stability and $\ell_2$-gain performance of recurrent neural networks (RNNs) with ReLU activation functions. These conditions are derived by combining Lyapunov/dissipativity theory with Quadratic Constraints (QCs) satisfied by repeated ReLUs. We write a general class of QCs for repeated RELUs using known properties for the scalar ReLU. Our stability and performance condition uses these QCs along with a "lifted" representation for the ReLU RNN. We show that the positive homogeneity property satisfied by a scalar ReLU does not expand the class of QCs for the repeated ReLU. We present examples to demonstrate the stability / performance condition and study the effect of the lifting horizon.