"I'm" Lost in Translation: Pronoun Missteps in Crowdsourced Data Sets
This work addresses translation biases in multilingual NLP datasets, which is crucial for improving natural communication in virtual assistants globally, though it is incremental in scope.
The study identified masculine pronoun biases in English-Japanese translations within the Tatoeba crowdsourced dataset, highlighting issues with feminine, neutral, and non-binary pronouns, and proposed a solution to embed plurality in NLP datasets.
As virtual assistants continue to be taken up globally, there is an ever-greater need for these speech-based systems to communicate naturally in a variety of languages. Crowdsourcing initiatives have focused on multilingual translation of big, open data sets for use in natural language processing (NLP). Yet, language translation is often not one-to-one, and biases can trickle in. In this late-breaking work, we focus on the case of pronouns translated between English and Japanese in the crowdsourced Tatoeba database. We found that masculine pronoun biases were present overall, even though plurality in language was accounted for in other ways. Importantly, we detected biases in the translation process that reflect nuanced reactions to the presence of feminine, neutral, and/or non-binary pronouns. We raise the issue of translation bias for pronouns and offer a practical solution to embed plurality in NLP data sets.