In-context Learning as Maintaining Coherency: A Study of On-the-fly Machine Translation Using Large Language Models
This work addresses on-the-fly adaptation in machine translation for users needing quick domain-specific translations, but it is incremental as it builds on existing in-context learning concepts.
The study investigated in-context learning for machine translation by framing it as maintaining coherency with prompt examples, finding that translation performance improves with in-domain prompts and long-term coherency, as demonstrated across three models and three translation directions.
The phenomena of in-context learning has typically been thought of as "learning from examples". In this work which focuses on Machine Translation, we present a perspective of in-context learning as the desired generation task maintaining coherency with its context, i.e., the prompt examples. We first investigate randomly sampled prompts across 4 domains, and find that translation performance improves when shown in-domain prompts. Next, we investigate coherency for the in-domain setting, which uses prompt examples from a moving window. We study this with respect to other factors that have previously been identified in the literature such as length, surface similarity and sentence embedding similarity. Our results across 3 models (GPTNeo2.7B, Bloom3B, XGLM2.9B), and three translation directions (\texttt{en}$\rightarrow$\{\texttt{pt, de, fr}\}) suggest that the long-term coherency of the prompts and the test sentence is a good indicator of downstream translation performance. In doing so, we demonstrate the efficacy of In-context Machine Translation for on-the-fly adaptation.