Convex Programming for Estimation in Nonlinear Recurrent Models
This work addresses estimation challenges in nonlinear recurrent models for researchers in machine learning and control theory, but it appears incremental as it builds on existing convex optimization frameworks.
The authors tackled the problem of parameter estimation in nonlinear recurrent models, including recurrent neural networks, by formulating it as a convex program, and they provided a sample complexity analysis under stability and regularity conditions, with simulation results on synthetic data suggesting potential relaxation of these assumptions.
We propose a formulation for nonlinear recurrent models that includes simple parametric models of recurrent neural networks as a special case. The proposed formulation leads to a natural estimator in the form of a convex program. We provide a sample complexity for this estimator in the case of stable dynamics, where the nonlinear recursion has a certain contraction property, and under certain regularity conditions on the input distribution. We evaluate the performance of the estimator by simulation on synthetic data. These numerical experiments also suggest the extent at which the imposed theoretical assumptions may be relaxed.