Implicitly Constrained Semi-Supervised Least Squares Classification
This provides a semi-supervised method for classification tasks that avoids explicit assumptions, but it is incremental as it builds on existing least squares classifiers without broad SOTA impact.
The paper tackles the problem of semi-supervised classification by introducing an implicitly constrained least squares (ICLS) classifier that minimizes squared loss on labeled data while considering all possible labelings of unlabeled data, and proves it never performs worse than the supervised classifier, with experimental results on benchmark datasets showing competitive error rates.
We introduce a novel semi-supervised version of the least squares classifier. This implicitly constrained least squares (ICLS) classifier minimizes the squared loss on the labeled data among the set of parameters implied by all possible labelings of the unlabeled data. Unlike other discriminative semi-supervised methods, our approach does not introduce explicit additional assumptions into the objective function, but leverages implicit assumptions already present in the choice of the supervised least squares classifier. We show this approach can be formulated as a quadratic programming problem and its solution can be found using a simple gradient descent procedure. We prove that, in a certain way, our method never leads to performance worse than the supervised classifier. Experimental results corroborate this theoretical result in the multidimensional case on benchmark datasets, also in terms of the error rate.