XLST: Cross-lingual Self-training to Learn Multilingual Representation for Low Resource Speech Recognition
This addresses the problem of limited annotated data for low-resource languages in speech recognition, offering a weakly supervised approach that leverages high-resource language data, though it is incremental as it builds on existing self-training and augmentation techniques.
The paper tackles low-resource speech recognition by proposing XLST, a cross-lingual self-training framework that uses a small amount of annotated high-resource language data to improve multilingual representation learning, achieving an 18.6% relative PER reduction over state-of-the-art methods on 5 low-resource ASR tasks.
In this paper, we propose a weakly supervised multilingual representation learning framework, called cross-lingual self-training (XLST). XLST is able to utilize a small amount of annotated data from high-resource languages to improve the representation learning on multilingual un-annotated data. Specifically, XLST uses a supervised trained model to produce initial representations and another model to learn from them, by maximizing the similarity between output embeddings of these two models. Furthermore, the moving average mechanism and multi-view data augmentation are employed, which are experimentally shown to be crucial to XLST. Comprehensive experiments have been conducted on the CommonVoice corpus to evaluate the effectiveness of XLST. Results on 5 downstream low-resource ASR tasks shows that our multilingual pretrained model achieves relatively 18.6% PER reduction over the state-of-the-art self-supervised method, with leveraging additional 100 hours of annotated English data.