Evaluation of Google Translate for Mandarin Chinese translation using sentiment and semantic analysis
This provides an incremental assessment of translation quality for users relying on automated tools for Mandarin Chinese, highlighting limitations in cultural and contextual accuracy.
The study evaluated Google Translate's Mandarin Chinese to English translation quality using sentiment and semantic analysis on the novel 'The True Story of Ah Q', finding that it differs in precision compared to human expert translations, particularly in handling specific words like Chinese traditional allusions.
Machine translation using large language models (LLMs) is having a significant global impact, making communication easier. Mandarin Chinese is the official language used for communication by the government and media in China. In this study, we provide an automated assessment of translation quality of Google Translate with human experts using sentiment and semantic analysis. In order to demonstrate our framework, we select the classic early twentieth-century novel 'The True Story of Ah Q' with selected Mandarin Chinese to English translations. We use Google Translate to translate the given text into English and then conduct a chapter-wise sentiment analysis and semantic analysis to compare the extracted sentiments across the different translations. Our results indicate that the precision of Google Translate differs both in terms of semantic and sentiment analysis when compared to human expert translations. We find that Google Translate is unable to translate some of the specific words or phrases in Chinese, such as Chinese traditional allusions. The mistranslations may be due to lack of contextual significance and historical knowledge of China.