Prediction-Powered Semi-Supervised Learning with Online Power Tuning
This work addresses a key problem in semi-supervised learning for practitioners by mitigating bias from poor pseudo-labels, though it is incremental as it builds on existing PPI techniques.
The paper tackles the challenge of bias from inaccurate pseudo-labels in semi-supervised learning by extending Prediction-Powered Inference to create an unbiased gradient estimator and tuning an interpolation parameter online, resulting in improved performance over classic SSL baselines and offline PPI methods.
Prediction-Powered Inference (PPI) is a recently proposed statistical inference technique for parameter estimation that leverages pseudo-labels on both labeled and unlabeled data to construct an unbiased, low-variance estimator. In this work, we extend its core idea to semi-supervised learning (SSL) for model training, introducing a novel unbiased gradient estimator. This extension addresses a key challenge in SSL: while unlabeled data can improve model performance, its benefit heavily depends on the quality of pseudo-labels. Inaccurate pseudo-labels can introduce bias, leading to suboptimal models.To balance the contributions of labeled and pseudo-labeled data, we utilize an interpolation parameter and tune it on the fly, alongside the model parameters, using a one-dimensional online learning algorithm. We verify the practical advantage of our approach through experiments on both synthetic and real datasets, demonstrating improved performance over classic SSL baselines and PPI methods that tune the interpolation parameter offline.