Measuring Sentiment Bias in Machine Translation
This addresses concerns about bias in generative models for machine translation users, but results are incremental as no significant bias was detected.
The study investigated whether machine translation introduces sentiment bias by comparing three models across five languages on two corpora, finding no consistent bias despite shifts in sentiment label distributions.
Biases induced to text by generative models have become an increasingly large topic in recent years. In this paper we explore how machine translation might introduce a bias in sentiments as classified by sentiment analysis models. For this, we compare three open access machine translation models for five different languages on two parallel corpora to test if the translation process causes a shift in sentiment classes recognized in the texts. Though our statistic test indicate shifts in the label probability distributions, we find none that appears consistent enough to assume a bias induced by the translation process.