Control on the Manifolds of Mappings with a View to the Deep Learning
This work offers a novel theoretical perspective on deep learning as a control problem, potentially impacting researchers in machine learning theory.
The authors tackled the problem of approximating desired input-output maps in deep learning by proposing a continuous-time control system as an approximant, resulting in a framework where neural networks are viewed as networks with a continuum of layers parameterized by time.
Deep learning of the Artificial Neural Networks (ANN) can be treated as a particular class of interpolation problems. The goal is to find a neural network whose input-output map approximates well the desired map on a finite or an infinite training set. Our idea consists of taking as an approximant the input-output map, which arises from a nonlinear continuous-time control system. In the limit such control system can be seen as a network with a continuum of layers, each one labelled by the time variable. The values of the controls at each instant of time are the parameters of the layer.