Toward Machine Translation Literacy: How Lay Users Perceive and Rely on Imperfect Translations
This research addresses the problem of over-reliance on imperfect machine translation among lay users, which is incremental as it builds on existing MT evaluation studies by focusing on real-world casual use.
The study investigated how bilingual and non-bilingual users perceive and rely on imperfect machine translations in a public museum setting, finding that non-bilingual users often over-rely on MT due to limited evaluation strategies, and experiencing errors can lead them to reassess future reliance.
As Machine Translation (MT) becomes increasingly commonplace, understanding how the general public perceives and relies on imperfect MT is crucial for contextualizing MT research in real-world applications. We present a human study conducted in a public museum (n=452), investigating how fluency and adequacy errors impact bilingual and non-bilingual users' reliance on MT during casual use. Our findings reveal that non-bilingual users often over-rely on MT due to a lack of evaluation strategies and alternatives, while experiencing the impact of errors can prompt users to reassess future reliance. This highlights the need for MT evaluation and NLP explanation techniques to promote not only MT quality, but also MT literacy among its users.