Finetuning End-to-End Models for Estonian Conversational Spoken Language Translation
This addresses the challenge of conversational speech-to-text translation for low-resource languages like Estonian, but it is incremental as it builds on existing models with synthetic data.
The paper tackled the problem of limited speech translation data for Estonian by creating synthetic data through web scraping and machine translation, and fine-tuned end-to-end models like Whisper, OWSM 3.1, and SeamlessM4T, resulting in SeamlessM4T matching or surpassing cascaded systems in translation accuracy.
This paper investigates the finetuning of end-to-end models for bidirectional Estonian-English and Estonian-Russian conversational speech-to-text translation. Due to the limited availability of speech translation data for Estonian, we created additional training data by web scraping and synthesizing data from speech recognition datasets using machine translation. We evaluated three publicly available end-to-end models: Whisper, OWSM 3.1, and SeamlessM4T. Our results indicate that fine-tuning with synthetic data enhances translation accuracy by a large margin, with SeamlessM4T matching or surpassing cascaded speech translation systems that use state-of-the-art speech recognition and machine translation models.