The IWSLT 2021 BUT Speech Translation Systems
This work addresses speech translation efficiency for language processing applications, but it is incremental as it builds on existing joint ASR-MT models with specific optimizations.
The paper tackles the problem of improving English to German offline speech translation by leveraging large amounts of separate ASR and MT training data alongside limited speech-translation data, resulting in significant improvements through joint training and punctuation handling in the ASR module.
The paper describes BUT's English to German offline speech translation(ST) systems developed for IWSLT2021. They are based on jointly trained Automatic Speech Recognition-Machine Translation models. Their performances is evaluated on MustC-Common test set. In this work, we study their efficiency from the perspective of having a large amount of separate ASR training data and MT training data, and a smaller amount of speech-translation training data. Large amounts of ASR and MT training data are utilized for pre-training the ASR and MT models. Speech-translation data is used to jointly optimize ASR-MT models by defining an end-to-end differentiable path from speech to translations. For this purpose, we use the internal continuous representations from the ASR-decoder as the input to MT module. We show that speech translation can be further improved by training the ASR-decoder jointly with the MT-module using large amount of text-only MT training data. We also show significant improvements by training an ASR module capable of generating punctuated text, rather than leaving the punctuation task to the MT module.