Towards the Effect of Examples on In-Context Learning: A Theoretical Case Study
This work provides theoretical insights into ICL mechanisms, addressing a fundamental gap in understanding LLM behaviors for researchers in machine learning and AI, though it is incremental as it builds on existing ICL studies with a specific case focus.
The paper tackles the problem of understanding how in-context learning (ICL) in large language models integrates pre-training knowledge with demonstration examples, using a theoretical study on binary classification tasks. It finds that the reliance on pre-training knowledge versus examples depends on the number of examples, and that label frequency and noise affect prediction accuracy, with the minor class having lower accuracy and noise impact varying by class-specific levels.
In-context learning (ICL) has emerged as a powerful capability for large language models (LLMs) to adapt to downstream tasks by leveraging a few (demonstration) examples. Despite its effectiveness, the mechanism behind ICL remains underexplored. To better understand how ICL integrates the examples with the knowledge learned by the LLM during pre-training (i.e., pre-training knowledge) and how the examples impact ICL, this paper conducts a theoretical study in binary classification tasks. In particular, we introduce a probabilistic model extending from the Gaussian mixture model to exactly quantify the impact of pre-training knowledge, label frequency, and label noise on the prediction accuracy. Based on our analysis, when the pre-training knowledge contradicts the knowledge in the examples, whether ICL prediction relies more on the pre-training knowledge or the examples depends on the number of examples. In addition, the label frequency and label noise of the examples both affect the accuracy of the ICL prediction, where the minor class has a lower accuracy, and how the label noise impacts the accuracy is determined by the specific noise level of the two classes. Extensive simulations are conducted to verify the correctness of the theoretical results, and real-data experiments also align with the theoretical insights. Our work reveals the role of pre-training knowledge and examples in ICL, offering a deeper understanding of LLMs' behaviors in classification tasks.