Gender in Danger? Evaluating Speech Translation Technology on the MuST-SHE Corpus
This addresses gender bias in speech translation for languages with grammatical gender, offering a new benchmark for future research, though it is incremental as it builds on existing translation technologies.
The study tackled gender bias in speech translation by evaluating cascade and end-to-end technologies on English-Italian/French directions, finding that audio input can provide additional information to reduce bias, with specific performance metrics reported in the benchmark.
Translating from languages without productive grammatical gender like English into gender-marked languages is a well-known difficulty for machines. This difficulty is also due to the fact that the training data on which models are built typically reflect the asymmetries of natural languages, gender bias included. Exclusively fed with textual data, machine translation is intrinsically constrained by the fact that the input sentence does not always contain clues about the gender identity of the referred human entities. But what happens with speech translation, where the input is an audio signal? Can audio provide additional information to reduce gender bias? We present the first thorough investigation of gender bias in speech translation, contributing with: i) the release of a benchmark useful for future studies, and ii) the comparison of different technologies (cascade and end-to-end) on two language directions (English-Italian/French).