Deep Learning via Dynamical Systems: An Approximation Perspective
This work contributes to building a mathematical framework for investigating deep learning, offering foundational insights for researchers in approximation theory and machine learning.
The paper tackles the problem of understanding deep residual networks as continuous-time dynamical systems and establishes sufficient conditions for universal approximation, including specific cases with approximation rates in terms of time horizon.
We build on the dynamical systems approach to deep learning, where deep residual networks are idealized as continuous-time dynamical systems, from the approximation perspective. In particular, we establish general sufficient conditions for universal approximation using continuous-time deep residual networks, which can also be understood as approximation theories in $L^p$ using flow maps of dynamical systems. In specific cases, rates of approximation in terms of the time horizon are also established. Overall, these results reveal that composition function approximation through flow maps present a new paradigm in approximation theory and contributes to building a useful mathematical framework to investigate deep learning.