Robust Semi-supervised Learning via $f$-Divergence and $α$-Rényi Divergence
It addresses robustness in semi-supervised learning for scenarios with noisy pseudo-labels, but appears incremental as it builds on existing divergence-based approaches.
This paper tackles the problem of noisy pseudo-labels in semi-supervised learning by proposing empirical risk functions and regularization methods based on f-divergences and α-Rényi divergences, showing better performance than traditional self-training methods under certain conditions.
This paper investigates a range of empirical risk functions and regularization methods suitable for self-training methods in semi-supervised learning. These approaches draw inspiration from various divergence measures, such as $f$-divergences and $α$-Rényi divergences. Inspired by the theoretical foundations rooted in divergences, i.e., $f$-divergences and $α$-Rényi divergence, we also provide valuable insights to enhance the understanding of our empirical risk functions and regularization techniques. In the pseudo-labeling and entropy minimization techniques as self-training methods for effective semi-supervised learning, the self-training process has some inherent mismatch between the true label and pseudo-label (noisy pseudo-labels) and some of our empirical risk functions are robust, concerning noisy pseudo-labels. Under some conditions, our empirical risk functions demonstrate better performance when compared to traditional self-training methods.