LLM-to-Speech: A Synthetic Data Pipeline for Training Dialectal Text-to-Speech Models
This addresses the lack of resources for Egyptian Arabic, the most widely understood Arabic dialect, in speech synthesis research.
The paper tackles the under-resourcing of Egyptian Arabic in text-to-speech (TTS) by introducing NileTTS, a 38-hour transcribed speech dataset from two speakers across diverse domains, and fine-tuning a state-of-the-art model to advance synthesis for this dialect.
Despite the advances in neural text to speech (TTS), many Arabic dialectal varieties remain marginally addressed, with most resources concentrated on Modern Spoken Arabic (MSA) and Gulf dialects, leaving Egyptian Arabic -- the most widely understood Arabic dialect -- severely under-resourced. We address this gap by introducing NileTTS: 38 hours of transcribed speech from two speakers across diverse domains including medical, sales, and general conversations. We construct this dataset using a novel synthetic pipeline: large language models (LLM) generate Egyptian Arabic content, which is then converted to natural speech using audio synthesis tools, followed by automatic transcription and speaker diarization with manual quality verification. We fine-tune XTTS v2, a state-of-the-art multilingual TTS model, on our dataset and evaluate against the baseline model trained on other Arabic dialects. Our contributions include: (1) the first publicly available Egyptian Arabic TTS dataset, (2) a reproducible synthetic data generation pipeline for dialectal TTS, and (3) an open-source fine-tuned model. All resources are released to advance Egyptian Arabic speech synthesis research.