Optimistic Semi-supervised Least Squares Classification
This work addresses semi-supervised learning for classification, but it is incremental as it adapts existing self-learning methods to a specific classifier.
The paper tackles the problem of improving supervised classifiers in semi-supervised learning by applying self-learning to the least squares classifier, showing that a soft-label variant outperforms a hard-label variant on benchmark datasets.
The goal of semi-supervised learning is to improve supervised classifiers by using additional unlabeled training examples. In this work we study a simple self-learning approach to semi-supervised learning applied to the least squares classifier. We show that a soft-label and a hard-label variant of self-learning can be derived by applying block coordinate descent to two related but slightly different objective functions. The resulting soft-label approach is related to an idea about dealing with missing data that dates back to the 1930s. We show that the soft-label variant typically outperforms the hard-label variant on benchmark datasets and partially explain this behaviour by studying the relative difficulty of finding good local minima for the corresponding objective functions.