In-Context Learning with Noisy Labels
This addresses a practical issue for real-world applications of in-context learning, but it is incremental as it builds on existing noisy label learning techniques.
The paper tackles the problem of noisy labels in demonstrations for in-context learning of large language models, proposing a new method that mitigates performance degradation caused by label corruption.
In-context learning refers to the emerging ability of large language models (LLMs) to perform a target task without additional training, utilizing demonstrations of the task. Recent studies aim to enhance in-context learning performance by selecting more useful demonstrations. However, they overlook the presence of inevitable noisy labels in task demonstrations that arise during the labeling process in the real-world. In this paper, we propose a new task, in-context learning with noisy labels, which aims to solve real-world problems for in-context learning where labels in task demonstrations would be corrupted. Moreover, we propose a new method and baseline methods for the new task, inspired by studies in learning with noisy labels. Through experiments, we demonstrate that our proposed method can serve as a safeguard against performance degradation in in-context learning caused by noisy labels.