Classifier-guided CLIP Distillation for Unsupervised Multi-label Classification
This work addresses the challenge of costly annotations in multi-label image classification for computer vision applications, presenting an incremental improvement over prior CLIP-based methods.
The paper tackles the problem of unsupervised multi-label classification by addressing CLIP's view-dependent predictions and bias, proposing a method that uses classifier-guided views and debiasing to improve performance, with experimental results showing superiority over existing techniques.
Multi-label classification is crucial for comprehensive image understanding, yet acquiring accurate annotations is challenging and costly. To address this, a recent study suggests exploiting unsupervised multi-label classification leveraging CLIP, a powerful vision-language model. Despite CLIP's proficiency, it suffers from view-dependent predictions and inherent bias, limiting its effectiveness. We propose a novel method that addresses these issues by leveraging multiple views near target objects, guided by Class Activation Mapping (CAM) of the classifier, and debiasing pseudo-labels derived from CLIP predictions. Our Classifier-guided CLIP Distillation (CCD) enables selecting multiple local views without extra labels and debiasing predictions to enhance classification performance. Experimental results validate our method's superiority over existing techniques across diverse datasets. The code is available at https://github.com/k0u-id/CCD.