Generalization in Machine Learning via Analytical Learning Theory
This provides a new theoretical foundation for regularization in deep learning, impacting researchers and practitioners by offering insights into generalization without relying on statistical assumptions.
The paper tackles the problem of generalization in machine learning by introducing a measure-theoretic theory that avoids statistical assumptions, and it shows that a derived regularization method outperforms previous methods on datasets like CIFAR-10, CIFAR-100, and SVHN with concrete performance gains.
This paper introduces a novel measure-theoretic theory for machine learning that does not require statistical assumptions. Based on this theory, a new regularization method in deep learning is derived and shown to outperform previous methods in CIFAR-10, CIFAR-100, and SVHN. Moreover, the proposed theory provides a theoretical basis for a family of practically successful regularization methods in deep learning. We discuss several consequences of our results on one-shot learning, representation learning, deep learning, and curriculum learning. Unlike statistical learning theory, the proposed learning theory analyzes each problem instance individually via measure theory, rather than a set of problem instances via statistics. As a result, it provides different types of results and insights when compared to statistical learning theory.