Contrastive Representations for Label Noise Require Fine-Tuning
This addresses label noise robustness in machine learning, but it is incremental as it builds on existing contrastive and noise-robust methods.
The paper tackles the problem of label noise in classification by showing that fine-tuning contrastive representations with noise-robust heads achieves state-of-the-art performance, outperforming frozen representations and end-to-end learning across various noise types and levels.
In this paper we show that the combination of a Contrastive representation with a label noise-robust classification head requires fine-tuning the representation in order to achieve state-of-the-art performances. Since fine-tuned representations are shown to outperform frozen ones, one can conclude that noise-robust classification heads are indeed able to promote meaningful representations if provided with a suitable starting point. Experiments are conducted to draw a comprehensive picture of performances by featuring six methods and nine noise instances of three different kinds (none, symmetric, and asymmetric). In presence of noise the experiments show that fine tuning of Contrastive representation allows the six methods to achieve better results than end-to-end learning and represent a new reference compare to the recent state of art. Results are also remarkable stable versus the noise level.