Beyond General Purpose Machine Translation: The Need for Context-specific Empirical Research to Design for Appropriate User Trust
This work addresses the challenge of designing for appropriate user trust in MT systems, which is critical for safe deployment in domains like healthcare, but it is incremental as it advocates for more empirical research rather than proposing a new solution.
The paper tackles the problem of users needing to calibrate trust in machine translation (MT) systems, especially in high-stakes scenarios like healthcare, by conducting interviews with 20 clinicians to understand their communication practices and MT usage.
Machine Translation (MT) has the potential to help people overcome language barriers and is widely used in high-stakes scenarios, such as in hospitals. However, in order to use MT reliably and safely, users need to understand when to trust MT outputs and how to assess the quality of often imperfect translation results. In this paper, we discuss research directions to support users to calibrate trust in MT systems. We share findings from an empirical study in which we conducted semi-structured interviews with 20 clinicians to understand how they communicate with patients across language barriers, and if and how they use MT systems. Based on our findings, we advocate for empirical research on how MT systems are used in practice as an important first step to addressing the challenges in building appropriate trust between users and MT tools.