End-to-End Speech Translation with Knowledge Distillation
This work addresses the challenge of combining speech recognition and translation into a single model for lower latency and error propagation, though it is incremental as it builds on existing knowledge distillation techniques.
The paper tackles the difficulty of end-to-end speech translation by proposing a knowledge distillation approach that transfers knowledge from a text translation teacher model to improve the ST model, resulting in significant improvements of over 3.5 BLEU points on English-French and English-Chinese datasets.
End-to-end speech translation (ST), which directly translates from source language speech into target language text, has attracted intensive attentions in recent years. Compared to conventional pipeline systems, end-to-end ST models have advantages of lower latency, smaller model size and less error propagation. However, the combination of speech recognition and text translation in one model is more difficult than each of these two tasks. In this paper, we propose a knowledge distillation approach to improve ST model by transferring the knowledge from text translation model. Specifically, we first train a text translation model, regarded as a teacher model, and then ST model is trained to learn output probabilities from teacher model through knowledge distillation. Experiments on English- French Augmented LibriSpeech and English-Chinese TED corpus show that end-to-end ST is possible to implement on both similar and dissimilar language pairs. In addition, with the instruction of teacher model, end-to-end ST model can gain significant improvements by over 3.5 BLEU points.