ArTST: Arabic Text and Speech Transformer
This addresses the problem of limited speech technology resources for Arabic speakers, though it is incremental as it adapts an existing English framework to Arabic.
The authors tackled the lack of open-source speech technologies for Arabic by developing ArTST, a pre-trained Arabic text and speech transformer, which performed on par with or exceeded state-of-the-art results in automatic speech recognition, text-to-speech synthesis, and spoken dialect identification.
We present ArTST, a pre-trained Arabic text and speech transformer for supporting open-source speech technologies for the Arabic language. The model architecture follows the unified-modal framework, SpeechT5, that was recently released for English, and is focused on Modern Standard Arabic (MSA), with plans to extend the model for dialectal and code-switched Arabic in future editions. We pre-trained the model from scratch on MSA speech and text data, and fine-tuned it for the following tasks: Automatic Speech Recognition (ASR), Text-To-Speech synthesis (TTS), and spoken dialect identification. In our experiments comparing ArTST with SpeechT5, as well as with previously reported results in these tasks, ArTST performs on a par with or exceeding the current state-of-the-art in all three tasks. Moreover, we find that our pre-training is conducive for generalization, which is particularly evident in the low-resource TTS task. The pre-trained model as well as the fine-tuned ASR and TTS models are released for research use.