Asymptotic Bayes risk for Gaussian mixture in a semi-supervised setting
This provides theoretical insights into semi-supervised learning benefits, but is incremental as it applies existing tools to a specific model.
The paper analytically computes the performance gap between fully-supervised and semi-supervised learning for a high-dimensional Gaussian mixture model, quantifying the accuracy increase achievable from unlabeled data.
Semi-supervised learning (SSL) uses unlabeled data for training and has been shown to greatly improve performance when compared to a supervised approach on the labeled data available. This claim depends both on the amount of labeled data available and on the algorithm used. In this paper, we compute analytically the gap between the best fully-supervised approach using only labeled data and the best semi-supervised approach using both labeled and unlabeled data. We quantify the best possible increase in performance obtained thanks to the unlabeled data, i.e. we compute the accuracy increase due to the information contained in the unlabeled data. Our work deals with a simple high-dimensional Gaussian mixture model for the data in a Bayesian setting. Our rigorous analysis builds on recent theoretical breakthroughs in high-dimensional inference and a large body of mathematical tools from statistical physics initially developed for spin glasses.