Enhancing Semi-Supervised Learning via Representative and Diverse Sample Selection
This addresses the under-explored issue of sample selection for annotation in semi-supervised learning, particularly beneficial for applications with constrained labeling budgets.
The paper tackles the problem of sample selection in semi-supervised learning under low-budget settings by proposing a Representative and Diverse Sample Selection (RDSS) approach, which improves performance by consistently outperforming state-of-the-art methods in active learning and semi-supervised active learning.
Semi-Supervised Learning (SSL) has become a preferred paradigm in many deep learning tasks, which reduces the need for human labor. Previous studies primarily focus on effectively utilising the labelled and unlabeled data to improve performance. However, we observe that how to select samples for labelling also significantly impacts performance, particularly under extremely low-budget settings. The sample selection task in SSL has been under-explored for a long time. To fill in this gap, we propose a Representative and Diverse Sample Selection approach (RDSS). By adopting a modified Frank-Wolfe algorithm to minimise a novel criterion $α$-Maximum Mean Discrepancy ($α$-MMD), RDSS samples a representative and diverse subset for annotation from the unlabeled data. We demonstrate that minimizing $α$-MMD enhances the generalization ability of low-budget learning. Experimental results show that RDSS consistently improves the performance of several popular SSL frameworks and outperforms the state-of-the-art sample selection approaches used in Active Learning (AL) and Semi-Supervised Active Learning (SSAL), even with constrained annotation budgets.