Learning with Noisy Labels Using Collaborative Sample Selection and Contrastive Semi-Supervised Learning
This work addresses the challenge of noisy labels in machine learning, which is critical for real-world applications where data annotation is imperfect, but it is incremental as it builds on existing LNL frameworks by integrating pre-trained models and co-training mechanisms.
The paper tackles the problem of learning with noisy labels by addressing the limitation of existing methods where selected clean sets contain noisy samples, leading to confirmation bias and impaired generalization; the proposed method, Collaborative Sample Selection with contrastive semi-supervised learning, leverages CLIP and a 2D-GMM to remove noisy samples, resulting in improved performance on benchmark datasets compared to state-of-the-art approaches.
Learning with noisy labels (LNL) has been extensively studied, with existing approaches typically following a framework that alternates between clean sample selection and semi-supervised learning (SSL). However, this approach has a limitation: the clean set selected by the Deep Neural Network (DNN) classifier, trained through self-training, inevitably contains noisy samples. This mixture of clean and noisy samples leads to misguidance in DNN training during SSL, resulting in impaired generalization performance due to confirmation bias caused by error accumulation in sample selection. To address this issue, we propose a method called Collaborative Sample Selection (CSS), which leverages the large-scale pre-trained model CLIP. CSS aims to remove the mixed noisy samples from the identified clean set. We achieve this by training a 2-Dimensional Gaussian Mixture Model (2D-GMM) that combines the probabilities from CLIP with the predictions from the DNN classifier. To further enhance the adaptation of CLIP to LNL, we introduce a co-training mechanism with a contrastive loss in semi-supervised learning. This allows us to jointly train the prompt of CLIP and the DNN classifier, resulting in improved feature representation, boosted classification performance of DNNs, and reciprocal benefits to our Collaborative Sample Selection. By incorporating auxiliary information from CLIP and utilizing prompt fine-tuning, we effectively eliminate noisy samples from the clean set and mitigate confirmation bias during training. Experimental results on multiple benchmark datasets demonstrate the effectiveness of our proposed method in comparison with the state-of-the-art approaches.