Solving a Machine Learning Regression Problem Based on the Theory of Random Functions
This provides a theoretical foundation for smoothing and interpolation methods, demonstrating optimality without a priori information, which is foundational for ML/AI.
The paper tackled the machine learning regression problem by deriving a regression method from postulates of indifference using the theory of random functions, showing that natural symmetries in a probability measure lead analytically to a kernel form, regularization, and noise parameterization, resulting in a generalized polyharmonic spline kernel.
This paper studies a machine learning regression problem as a multivariate approximation problem using the framework of the theory of random functions. An ab initio derivation of a regression method is proposed, starting from postulates of indifference. It is shown that if a probability measure on an infinite-dimensional function space possesses natural symmetries (invariance under translation, rotation, scaling, and Gaussianity), then the entire solution scheme, including the kernel form, the type of regularization, and the noise parameterization, follows analytically from these postulates. The resulting kernel coincides with a generalized polyharmonic spline; however, unlike existing approaches, it is not chosen empirically but arises as a consequence of the indifference principle. This result provides a theoretical foundation for a broad class of smoothing and interpolation methods, demonstrating their optimality in the absence of a priori information.