Understanding the Generalization of In-Context Learning in Transformers: An Empirical Study
This addresses the problem of limited understanding of ICL generalization for LLM developers, though it is incremental as it builds on existing empirical frameworks.
The study investigated the generalization boundaries of in-context learning in transformers, finding that they lack inter-problem generalization but excel in intra-task and intra-problem generalization, with training data diversity enhancing generalization on unseen tasks.
Large language models (LLMs) like GPT-4 and LLaMA-3 utilize the powerful in-context learning (ICL) capability of Transformer architecture to learn on the fly from limited examples. While ICL underpins many LLM applications, its full potential remains hindered by a limited understanding of its generalization boundaries and vulnerabilities. We present a systematic investigation of transformers' generalization capability with ICL relative to training data coverage by defining a task-centric framework along three dimensions: inter-problem, intra-problem, and intra-task generalization. Through extensive simulation and real-world experiments, encompassing tasks such as function fitting, API calling, and translation, we find that transformers lack inter-problem generalization with ICL, but excel in intra-task and intra-problem generalization. When the training data includes a greater variety of mixed tasks, it significantly enhances the generalization ability of ICL on unseen tasks and even on known simple tasks. This guides us in designing training data to maximize the diversity of tasks covered and to combine different tasks whenever possible, rather than solely focusing on the target task for testing.