Thinking Slow about Latency Evaluation for Simultaneous Machine Translation
This addresses the need for interpretable latency metrics in simultaneous translation systems, which is crucial for applications like live streaming and conversation, but it appears incremental as it modifies an existing metric.
The paper tackles the problem of evaluating latency in simultaneous machine translation by introducing Differentiable Average Lagging (DAL), a variant of Average Lagging that is differentiable and mathematically consistent, though no concrete performance numbers are provided.
Simultaneous machine translation attempts to translate a source sentence before it is finished being spoken, with applications to translation of spoken language for live streaming and conversation. Since simultaneous systems trade quality to reduce latency, having an effective and interpretable latency metric is crucial. We introduce a variant of the recently proposed Average Lagging (AL) metric, which we call Differentiable Average Lagging (DAL). It distinguishes itself by being differentiable and internally consistent to its underlying mathematical model.