Explaining GAND: A Resource on Gender-Ambiguous Natural Data & Contrastive Attribution
For MT researchers and practitioners, it provides a natural benchmark and interpretability method to diagnose and mitigate gender bias in translation systems.
The paper introduces GAND, a gender-ambiguous natural data benchmark for machine translation, and uses it with contrastive attribution to analyze how contextual cues influence gender translation, revealing source words that inform gender choices.
Machine translation (MT) systems continue to produce gender-biased translations. In a time where self-expression is paramount, mistranslations based on default behaviour and stereotyping can lead to harm for users of these systems. To better understand how these systems translate gender in the absence of clear gender cues, we need benchmarking resources that reflect gender-ambiguous scenarios in a natural way. To this end, we present GAND, a gender-ambiguous natural data benchmarking resource for MT consisting of English source sentences, specifically designed to analyse the influence of contextual cues on gender in translation. We leverage GAND to conduct an interpretability analysis: we translate a subset of GAND into two grammatical gender languages and extend these with manually crafted contrastive translations. A following feature attribution analysis reveals source words in context that inform the gender translation of an ambiguous referent entity in the target translation.