Decoding Machine Translationese in English-Chinese News: LLMs vs. NMTs
This work addresses the problem of linguistic peculiarities in machine translation for researchers and practitioners in natural language processing, though it is incremental as it extends existing MTese analysis to a new language pair and models.
This study investigated Machine Translationese (MTese) in English-to-Chinese news texts, confirming its presence in both Neural Machine Translation systems and Large Language Models, with original Chinese texts being nearly perfectly distinguishable from machine outputs and classification accuracy between LLMs and NMTs reaching approximately 70%.
This study explores Machine Translationese (MTese) -- the linguistic peculiarities of machine translation outputs -- focusing on the under-researched English-to-Chinese language pair in news texts. We construct a large dataset consisting of 4 sub-corpora and employ a comprehensive five-layer feature set. Then, a chi-square ranking algorithm is applied for feature selection in both classification and clustering tasks. Our findings confirm the presence of MTese in both Neural Machine Translation systems (NMTs) and Large Language Models (LLMs). Original Chinese texts are nearly perfectly distinguishable from both LLM and NMT outputs. Notable linguistic patterns in MT outputs are shorter sentence lengths and increased use of adversative conjunctions. Comparing LLMs and NMTs, we achieve approximately 70% classification accuracy, with LLMs exhibiting greater lexical diversity and NMTs using more brackets. Additionally, translation-specific LLMs show lower lexical diversity but higher usage of causal conjunctions compared to generic LLMs. Lastly, we find no significant differences between LLMs developed by Chinese firms and their foreign counterparts.