Addressing speaker gender bias in large scale speech translation systems
This addresses a fairness issue in speech translation for users affected by gender bias, representing a strong specific gain rather than a broad paradigm shift.
The study tackled speaker gender bias in speech translation systems, which causes offensive and inaccurate translations, by using LLMs to correct translations and fine-tuning the model to generate gender-specific translations from audio cues, resulting in a 70% improvement for female speakers compared to baselines.
This study addresses the issue of speaker gender bias in Speech Translation (ST) systems, which can lead to offensive and inaccurate translations. The masculine bias often found in large-scale ST systems is typically perpetuated through training data derived from Machine Translation (MT) systems. Our approach involves two key steps. First, we employ Large Language Models (LLMs) to rectify translations based on the speaker's gender in a cost-effective manner. Second, we fine-tune the ST model with the corrected data, enabling the model to generate gender-specific translations directly from audio cues, without the need for explicit gender input. Additionally, we propose a three-mode fine-tuned model for scenarios where the speaker's gender is either predefined or should not be inferred from speech cues. We demonstrate a 70% improvement in translations for female speakers compared to our baseline and other large-scale ST systems, such as Seamless M4T and Canary, on the MuST-SHE test set.