What about em? How Commercial Machine Translation Fails to Handle (Neo-)Pronouns
This addresses identity-inclusive NLP for marginalized groups like non-binary individuals, highlighting harmful translation failures in a widely used application.
The study examined how three commercial machine translation systems handle gendered and gender-neutral pronouns, including neopronouns, across multiple languages, finding that gender-neutral pronouns often cause grammatical and semantic errors and fail to preserve gender neutrality.
As 3rd-person pronoun usage shifts to include novel forms, e.g., neopronouns, we need more research on identity-inclusive NLP. Exclusion is particularly harmful in one of the most popular NLP applications, machine translation (MT). Wrong pronoun translations can discriminate against marginalized groups, e.g., non-binary individuals (Dev et al., 2021). In this ``reality check'', we study how three commercial MT systems translate 3rd-person pronouns. Concretely, we compare the translations of gendered vs. gender-neutral pronouns from English to five other languages (Danish, Farsi, French, German, Italian), and vice versa, from Danish to English. Our error analysis shows that the presence of a gender-neutral pronoun often leads to grammatical and semantic translation errors. Similarly, gender neutrality is often not preserved. By surveying the opinions of affected native speakers from diverse languages, we provide recommendations to address the issue in future MT research.