Semi-Supervised Learning guided by the Generalized Bayes Rule under Soft Revision
This work addresses robust pseudo-label selection for semi-supervised learning, offering incremental improvements in scenarios with limited labeled data.
The paper tackles the problem of pseudo-label selection in semi-supervised learning by proposing the Gamma-Maximin method with soft revision, which uses credal sets to handle epistemic uncertainty and selects pseudo-labels based on least favorable distributions. It shows that this method achieves promising results, particularly when labeled data is scarce, with concrete comparisons in logistic models.
We provide a theoretical and computational investigation of the Gamma-Maximin method with soft revision, which was recently proposed as a robust criterion for pseudo-label selection (PLS) in semi-supervised learning. Opposed to traditional methods for PLS we use credal sets of priors ("generalized Bayes") to represent the epistemic modeling uncertainty. These latter are then updated by the Gamma-Maximin method with soft revision. We eventually select pseudo-labeled data that are most likely in light of the least favorable distribution from the so updated credal set. We formalize the task of finding optimal pseudo-labeled data w.r.t. the Gamma-Maximin method with soft revision as an optimization problem. A concrete implementation for the class of logistic models then allows us to compare the predictive power of the method with competing approaches. It is observed that the Gamma-Maximin method with soft revision can achieve very promising results, especially when the proportion of labeled data is low.