Noise-Robust Fine-Tuning of Pretrained Language Models via External Guidance
This addresses the challenge of noisy data labels in real-world NLP applications, offering a method to improve model robustness, though it is incremental as it builds on existing fine-tuning paradigms.
The paper tackles the problem of fine-tuning pretrained language models with noisy labels by incorporating guidance from large language models like ChatGPT to distinguish clean from noisy samples and provide additional information, resulting in superior performance over state-of-the-art baselines on synthetic and real-world datasets.
Adopting a two-stage paradigm of pretraining followed by fine-tuning, Pretrained Language Models (PLMs) have achieved substantial advancements in the field of natural language processing. However, in real-world scenarios, data labels are often noisy due to the complex annotation process, making it essential to develop strategies for fine-tuning PLMs with such noisy labels. To this end, we introduce an innovative approach for fine-tuning PLMs using noisy labels, which incorporates the guidance of Large Language Models (LLMs) like ChatGPT. This guidance assists in accurately distinguishing between clean and noisy samples and provides supplementary information beyond the noisy labels, thereby boosting the learning process during fine-tuning PLMs. Extensive experiments on synthetic and real-world noisy datasets further demonstrate the superior advantages of our framework over the state-of-the-art baselines.