Supervised learning with probabilistic morphisms and kernel mean embeddings
This work addresses foundational theoretical issues in supervised learning, though it appears incremental as it builds upon existing results from Cucker-Smale and Vapnik-Stefanuyk.
The paper tackles two measurability problems in statistical learning theory by proposing convergence in outer probability to characterize learning algorithm consistency, and extends a regression learnability result to conditional probability estimation while presenting a regularization method to prove generalizability of overparameterized models.
In this paper I propose a generative model of supervised learning that unifies two approaches to supervised learning, using a concept of a correct loss function. Addressing two measurability problems, which have been ignored in statistical learning theory, I propose to use convergence in outer probability to characterize the consistency of a learning algorithm. Building upon these results, I extend a result due to Cucker-Smale, which addresses the learnability of a regression model, to the setting of a conditional probability estimation problem. Additionally, I present a variant of Vapnik-Stefanuyk's regularization method for solving stochastic ill-posed problems, and using it to prove the generalizability of overparameterized supervised learning models.