SiniticMTError: A Machine Translation Dataset with Error Annotations for Sinitic Languages
This provides a resource for the MT community to improve translation quality estimation and error-aware generation for low-resource languages, but it is incremental as it builds on existing corpora.
The authors tackled the lack of error-annotated data for low-resource Sinitic languages like Cantonese and Wu Chinese by introducing SiniticMTError, a dataset with error span, type, and severity annotations for English-to-Mandarin/Cantonese/Wu translations, reporting on inter-annotator agreement and error patterns.
Despite major advances in machine translation (MT) in recent years, progress remains limited for many low-resource languages that lack large-scale training data and linguistic resources. Cantonese and Wu Chinese are two Sinitic examples, although each enjoys more than 80 million speakers around the world. In this paper, we introduce SiniticMTError, a novel dataset that builds on existing parallel corpora to provide error span, error type, and error severity annotations in machine-translated examples from English to Mandarin, Cantonese, and Wu Chinese. Our dataset serves as a resource for the MT community to utilize in fine-tuning models with error detection capabilities, supporting research on translation quality estimation, error-aware generation, and low-resource language evaluation. We report our rigorous annotation process by native speakers, with analyses on inter-annotator agreement, iterative feedback, and patterns in error type and severity.