Competition Dynamics Shape Algorithmic Phases of In-Context Learning
This provides a mechanistic explanation for the transient nature of ICL, suggesting it is a mixture of algorithms rather than a monolithic capability, which is incremental for researchers in understanding and improving language models.
The paper tackles the lack of a unified framework for studying In-Context Learning (ICL) by proposing a synthetic sequence modeling task that reproduces key ICL results, revealing that ICL behavior is dictated by competition among four algorithms based on retrieval vs. inference and unigram or bigram statistics, with transitions driven by factors like context size.
In-Context Learning (ICL) has significantly expanded the general-purpose nature of large language models, allowing them to adapt to novel tasks using merely the inputted context. This has motivated a series of papers that analyze tractable synthetic domains and postulate precise mechanisms that may underlie ICL. However, the use of relatively distinct setups that often lack a sequence modeling nature to them makes it unclear how general the reported insights from such studies are. Motivated by this, we propose a synthetic sequence modeling task that involves learning to simulate a finite mixture of Markov chains. As we show, models trained on this task reproduce most well-known results on ICL, hence offering a unified setting for studying the concept. Building on this setup, we demonstrate we can explain a model's behavior by decomposing it into four broad algorithms that combine a fuzzy retrieval vs. inference approach with either unigram or bigram statistics of the context. These algorithms engage in a competition dynamics to dominate model behavior, with the precise experimental conditions dictating which algorithm ends up superseding others: e.g., we find merely varying context size or amount of training yields (at times sharp) transitions between which algorithm dictates the model behavior, revealing a mechanism that explains the transient nature of ICL. In this sense, we argue ICL is best thought of as a mixture of different algorithms, each with its own peculiarities, instead of a monolithic capability. This also implies that making general claims about ICL that hold universally across all settings may be infeasible.