Data Curation Alone Can Stabilize In-context Learning
This addresses the sensitivity of ICL for large language model users, offering a simple data-centric solution to improve reliability, though it is incremental as it builds on existing ICL methods.
The paper tackles the problem of high variance in in-context learning (ICL) performance due to random training example selection by showing that curating stable subsets of training data stabilizes ICL without algorithm changes, improving average accuracy by 7.7% and 6.3% across tasks and models.
In-context learning (ICL) enables large language models (LLMs) to perform new tasks by prompting them with a sequence of training examples. However, it is known that ICL is very sensitive to the choice of training examples: randomly sampling examples from a training set leads to high variance in performance. In this paper, we show that carefully curating a subset of training data greatly stabilizes ICL performance without any other changes to the ICL algorithm (e.g., prompt retrieval or calibration). We introduce two methods to choose training subsets -- both score training examples individually, then select the highest-scoring ones. CondAcc scores a training example by its average dev-set ICL accuracy when combined with random training examples, while Datamodels learns linear regressors that estimate how the presence of each training example influences LLM outputs. Across five tasks and two LLMs, sampling from stable subsets selected by CondAcc and Datamodels improves average accuracy over sampling from the entire training set by 7.7% and 6.3%, respectively. Surprisingly, the stable subset examples are not especially diverse in content or low in perplexity, in contrast with other work suggesting that diversity and perplexity are important when prompting LLMs.