Machine Learning from a Continuous Viewpoint
This work provides a foundational mathematical perspective for machine learning, potentially benefiting researchers by offering a unified framework to analyze generalization and regularization, though it appears incremental in extending classical numerical analysis concepts to ML.
The paper tackles the problem of unifying machine learning models under a continuous mathematical framework, showing that conventional models like random features and neural networks can be derived as discretizations of this formulation, and it introduces new models and algorithms such as flow-based random features and smoothed particle methods.
We present a continuous formulation of machine learning, as a problem in the calculus of variations and differential-integral equations, in the spirit of classical numerical analysis. We demonstrate that conventional machine learning models and algorithms, such as the random feature model, the two-layer neural network model and the residual neural network model, can all be recovered (in a scaled form) as particular discretizations of different continuous formulations. We also present examples of new models, such as the flow-based random feature model, and new algorithms, such as the smoothed particle method and spectral method, that arise naturally from this continuous formulation. We discuss how the issues of generalization error and implicit regularization can be studied under this framework.