Statistical Consistency of Finite-dimensional Unregularized Linear Classification
This work addresses the theoretical foundation for consistency in linear classification and boosting methods, providing incremental insights into finite-dimensional settings.
The paper tackles the problem of ensuring statistical consistency for finite-dimensional linear classifiers trained via unregularized convex risk minimization, showing that scaling the complexity of the weak learning class with sample size achieves optimal classification risk almost surely.
This manuscript studies statistical properties of linear classifiers obtained through minimization of an unregularized convex risk over a finite sample. Although the results are explicitly finite-dimensional, inputs may be passed through feature maps; in this way, in addition to treating the consistency of logistic regression, this analysis also handles boosting over a finite weak learning class with, for instance, the exponential, logistic, and hinge losses. In this finite-dimensional setting, it is still possible to fit arbitrary decision boundaries: scaling the complexity of the weak learning class with the sample size leads to the optimal classification risk almost surely.