ChatGPT as a Translation Engine: A Case Study on Japanese-English
This addresses translation quality for users of AI tools, but it is incremental as it applies existing methods to a new language pair.
This study tackled Japanese-English translation using ChatGPT, finding that document-level translation outperforms sentence-level, with ChatGPT-3.5 preferred for accuracy and ChatGPT-4 for fluency, yielding competitive results against commercial systems.
This study investigates ChatGPT for Japanese-English translation, exploring simple and enhanced prompts and comparing against commercially available translation engines. Performing both automatic and MQM-based human evaluations, we found that document-level translation outperforms sentence-level translation for ChatGPT. On the other hand, we were not able to determine if enhanced prompts performed better than simple prompts in our experiments. We also discovered that ChatGPT-3.5 was preferred by automatic evaluation, but a tradeoff exists between accuracy (ChatGPT-3.5) and fluency (ChatGPT-4). Lastly, ChatGPT yields competitive results against two widely-known translation systems.