Learning Recurrent Neural Net Models of Nonlinear Systems
This work provides theoretical guarantees for learning nonlinear system models using recurrent neural networks, which is important for researchers and practitioners in control systems and machine learning.
This paper addresses the problem of learning an unknown nonlinear system from input-output signal pairs. They derive quantitative guarantees on the sup-norm risk of a learned continuous-time recurrent neural net model, considering factors like neuron count, sample size, derivative matching order, and signal regularity.
We consider the following learning problem: Given sample pairs of input and output signals generated by an unknown nonlinear system (which is not assumed to be causal or time-invariant), we wish to find a continuous-time recurrent neural net with hyperbolic tangent activation function that approximately reproduces the underlying i/o behavior with high confidence. Leveraging earlier work concerned with matching output derivatives up to a given finite order, we reformulate the learning problem in familiar system-theoretic language and derive quantitative guarantees on the sup-norm risk of the learned model in terms of the number of neurons, the sample size, the number of derivatives being matched, and the regularity properties of the inputs, the outputs, and the unknown i/o map.