Fourier-domain Variational Formulation and Its Well-posedness for Supervised Learning
This work provides a theoretical framework for supervised learning, potentially benefiting researchers working on the mathematical foundations of machine learning, by offering a well-posed variational formulation.
This paper proposes a Fourier-domain variational formulation for supervised learning, which addresses the challenge of incorporating isolated data point constraints in continuous models. The authors establish the well-posedness of this formulation by identifying a critical exponent dependent on data dimension.
A supervised learning problem is to find a function in a hypothesis function space given values on isolated data points. Inspired by the frequency principle in neural networks, we propose a Fourier-domain variational formulation for supervised learning problem. This formulation circumvents the difficulty of imposing the constraints of given values on isolated data points in continuum modelling. Under a necessary and sufficient condition within our unified framework, we establish the well-posedness of the Fourier-domain variational problem, by showing a critical exponent depending on the data dimension. In practice, a neural network can be a convenient way to implement our formulation, which automatically satisfies the well-posedness condition.