Testing the Deliteralization Hypothesis in Human and Machine Translation
For translation studies and MT researchers, this work provides the first evidence that the deliteralization hypothesis applies to LLM generation, while revealing a fundamental difference in revision behavior between humans and LLMs.
The study tests the deliteralization hypothesis—that translations become less literal through revision—comparing human translations and post-editions with outputs from NMT systems and LLMs across 54 language pairs. Results show human translations are significantly less literal than MT outputs, though recent LLMs narrow the gap; LLMs deliteralize monotonically when iteratively revising their own output, but as post-editors they invert human revision triggers by tolerating literal drafts and targeting idiomatic human formulations.
The recent shift from dedicated NMT systems to general-purpose LLMs has reshaped machine translation, with LLMs reported to produce more fluent, less literal output than their predecessors. We test whether this shift extends to the deliteralization hypothesis, the long-standing claim from translation studies that translations become progressively less literal as they are drafted and revised. Using the WMT24++ dataset, we compare the literality of human translations and post-editions to that of two NMT systems and six LLMs across 54 language pairs and three tasks: direct translation, iterative self-revision, and post-editing of human drafts. Literality is measured via a validated Synthetic Literality Index built from six heuristics. We find that (i) human translations remain significantly less literal than those of all tested MT systems, though recent LLMs narrow the gap; (ii) when prompted to iteratively revise their own output, LLMs deliteralize monotonically, providing the first evidence that the hypothesis applies natively to LLM generation; and (iii) as post-editors, LLMs invert the revision triggers of human post-editors, tolerating literal drafts and targeting idiomatic human formulations for revision.