Transductive CLIP with Class-Conditional Contrastive Learning
This work addresses label noise in leveraging CLIP for data labeling, which is an incremental improvement for classification tasks.
The paper tackles the problem of noisy labels from CLIP supervision in classification by proposing Transductive CLIP, which uses class-conditional contrastive learning and ensemble labels to reduce noise impact, achieving substantial improvements on benchmark datasets.
Inspired by the remarkable zero-shot generalization capacity of vision-language pre-trained model, we seek to leverage the supervision from CLIP model to alleviate the burden of data labeling. However, such supervision inevitably contains the label noise, which significantly degrades the discriminative power of the classification model. In this work, we propose Transductive CLIP, a novel framework for learning a classification network with noisy labels from scratch. Firstly, a class-conditional contrastive learning mechanism is proposed to mitigate the reliance on pseudo labels and boost the tolerance to noisy labels. Secondly, ensemble labels is adopted as a pseudo label updating strategy to stabilize the training of deep neural networks with noisy labels. This framework can reduce the impact of noisy labels from CLIP model effectively by combining both techniques. Experiments on multiple benchmark datasets demonstrate the substantial improvements over other state-of-the-art methods.