Towards Speech Dialogue Translation Mediating Speakers of Different Languages
This addresses the problem of enabling real-time cross-lingual communication for users in dialogue settings, but it is incremental as it builds on existing models with a new dataset and context methods.
The paper introduces a new task of speech dialogue translation for mediating speakers of different languages, constructing the SpeechBSD dataset and showing that bilingual context outperforms monolingual context in baseline experiments using Whisper and mBART.
We present a new task, speech dialogue translation mediating speakers of different languages. We construct the SpeechBSD dataset for the task and conduct baseline experiments. Furthermore, we consider context to be an important aspect that needs to be addressed in this task and propose two ways of utilizing context, namely monolingual context and bilingual context. We conduct cascaded speech translation experiments using Whisper and mBART, and show that bilingual context performs better in our settings.