Generative Adversarial Training Data Adaptation for Very Low-resource Automatic Speech Recognition
This addresses the speaker sparsity issue in low-resource ASR for endangered language preservation, offering a practical adaptation method.
The paper tackled the problem of poor ASR performance for endangered languages due to limited speaker data by converting training speech to match the test speaker's voice using CycleGAN-based voice conversion, resulting in 35-60% relative improvement in phone error rate on Ainu and 40% on Mboshi corpora.
It is important to transcribe and archive speech data of endangered languages for preserving heritages of verbal culture and automatic speech recognition (ASR) is a powerful tool to facilitate this process. However, since endangered languages do not generally have large corpora with many speakers, the performance of ASR models trained on them are considerably poor in general. Nevertheless, we are often left with a lot of recordings of spontaneous speech data that have to be transcribed. In this work, for mitigating this speaker sparsity problem, we propose to convert the whole training speech data and make it sound like the test speaker in order to develop a highly accurate ASR system for this speaker. For this purpose, we utilize a CycleGAN-based non-parallel voice conversion technology to forge a labeled training data that is close to the test speaker's speech. We evaluated this speaker adaptation approach on two low-resource corpora, namely, Ainu and Mboshi. We obtained 35-60% relative improvement in phone error rate on the Ainu corpus, and 40% relative improvement was attained on the Mboshi corpus. This approach outperformed two conventional methods namely unsupervised adaptation and multilingual training with these two corpora.