Class Probability Estimation via Differential Geometric Regularization
This work addresses overfitting in classification tasks for machine learning practitioners, offering a novel regularization approach that is incremental in nature.
The authors tackled the problem of overfitting in supervised classification by proposing a differential geometric regularization technique that minimizes the volume of the class probability estimator's submanifold, leading to improved performance compared to standard regularization methods in binary and multiclass classification.
We study the problem of supervised learning for both binary and multiclass classification from a unified geometric perspective. In particular, we propose a geometric regularization technique to find the submanifold corresponding to a robust estimator of the class probability $P(y|\pmb{x})$. The regularization term measures the volume of this submanifold, based on the intuition that overfitting produces rapid local oscillations and hence large volume of the estimator. This technique can be applied to regularize any classification function that satisfies two requirements: firstly, an estimator of the class probability can be obtained; secondly, first and second derivatives of the class probability estimator can be calculated. In experiments, we apply our regularization technique to standard loss functions for classification, our RBF-based implementation compares favorably to widely used regularization methods for both binary and multiclass classification.