Learning from Imperfect Text Guidance: Robust Long-Tail Visual Recognition with High-Noise Label
For practitioners dealing with real-world long-tailed and noisy labeled data, this work provides a method that outperforms prior approaches in high-noise settings.
The paper addresses robust long-tail visual recognition under high-noise label conditions by proposing Weak Teacher Supervision (WTS), which leverages pre-trained vision-language models to correct label-image mismatches. WTS achieves superior performance on synthetic and real-world datasets, especially under high noise.
Real-world data often exhibit long-tailed distributions with numerous noisy labels, substantially degrading the performance of deep models. While prior research has made progress in addressing this combined challenge, it overlooks the severe label-image mismatch inherent to high-noise settings, thereby limiting their effectiveness. Given that observed labels, though mismatched with images, still retain category information, we propose employing auxiliary text information from labels to address label-image inconsistencies in long-tailed noisy data. Specifically, we leverage the intrinsic cross-modal alignment in pre-trained visual-language models to correct the label-image inconsistencies. This supervisory signal, referred to as Weak Teacher Supervision (WTS), is unaffected by label noise and data distribution biases, albeit exhibits limited accuracy. Therefore, the activation of WTS is determined by evaluating the discrepancy between text-predicted labels and observed labels. Extensive experiments demonstrate the superior performance of WTS across synthetic and real-world datasets, particularly under high-noise conditions. The source code is available at https://anonymous.4open.science/r/WTS-0F3C.