Large-Scale Self- and Semi-Supervised Learning for Speech Translation
This work addresses speech translation for multilingual applications, offering a simple and effective method that improves performance without additional supervision beyond standard ST data.
The paper tackles speech translation by leveraging large unlabeled speech and text data through pretraining and self-training, achieving an average improvement of 2.6 BLEU over previous state-of-the-art on CoVoST 2 language pairs.
In this paper, we improve speech translation (ST) through effectively leveraging large quantities of unlabeled speech and text data in different and complementary ways. We explore both pretraining and self-training by using the large Libri-Light speech audio corpus and language modeling with CommonCrawl. Our experiments improve over the previous state of the art by 2.6 BLEU on average on all four considered CoVoST 2 language pairs via a simple recipe of combining wav2vec 2.0 pretraining, a single iteration of self-training and decoding with a language model. Different to existing work, our approach does not leverage any other supervision than ST data. Code and models will be publicly released.