Metric-Entropy Limits on the Approximation of Nonlinear Dynamical Systems
This work addresses fundamental limits in approximating complex nonlinear systems, which is crucial for applications in control and time-series analysis, though it is incremental as it builds on existing approximation theory.
The paper tackled the problem of approximating nonlinear dynamical systems with recurrent neural networks (RNNs), showing that RNNs can achieve metric-entropy-optimal approximation for systems with Lipschitz properties and fast input forgetting, with results quantified in terms of order, type, and generalized dimension for exponentially- and polynomially Lipschitz fading-memory systems.
This paper is concerned with fundamental limits on the approximation of nonlinear dynamical systems. Specifically, we show that recurrent neural networks (RNNs) can approximate nonlinear systems -- that satisfy a Lipschitz property and forget past inputs fast enough -- in metric-entropy-optimal manner. As the sets of sequence-to-sequence mappings realized by the dynamical systems we consider are significantly more massive than function classes generally analyzed in approximation theory, a refined metric-entropy characterization is needed, namely in terms of order, type, and generalized dimension. We compute these quantities for the classes of exponentially- and polynomially Lipschitz fading-memory systems and show that RNNs can achieve them.