Voice Adaptation for Swiss German
This addresses the problem of voice adaptation for underrepresented languages like Swiss German, though it is incremental as it applies an existing method to new data.
This work tackled adapting voice cloning to Swiss German dialects by fine-tuning the XTTSv2 model on a large dataset of Swiss podcasts, achieving CMOS scores of up to -0.28 and SMOS scores of 3.8 in evaluations.
This work investigates the performance of Voice Adaptation models for Swiss German dialects, i.e., translating Standard German text to Swiss German dialect speech. For this, we preprocess a large dataset of Swiss podcasts, which we automatically transcribe and annotate with dialect classes, yielding approximately 5000 hours of weakly labeled training material. We fine-tune the XTTSv2 model on this dataset and show that it achieves good scores in human and automated evaluations and can correctly render the desired dialect. Our work shows a step towards adapting Voice Cloning technology to underrepresented languages. The resulting model achieves CMOS scores of up to -0.28 and SMOS scores of 3.8.