Exploring Machine Speech Chain for Domain Adaptation and Few-Shot Speaker Adaptation
This addresses domain mismatch in speech processing for applications like transcription and synthesis, but it is incremental as it builds on existing speech chain methods.
The paper tackled domain adaptation for speech models by using a machine speech chain with only target-domain text, achieving a 10% relative WER reduction for ASR and 51.5% for TTS when adapting from audiobook to presentation domains, and also applied few-shot speaker adaptation for further gains.
Machine Speech Chain, which integrates both end-to-end (E2E) automatic speech recognition (ASR) and text-to-speech (TTS) into one circle for joint training, has been proven to be effective in data augmentation by leveraging large amounts of unpaired data. In this paper, we explore the TTS->ASR pipeline in speech chain to do domain adaptation for both neural TTS and E2E ASR models, with only text data from target domain. We conduct experiments by adapting from audiobook domain (LibriSpeech) to presentation domain (TED-LIUM), there is a relative word error rate (WER) reduction of 10% for the E2E ASR model on the TED-LIUM test set, and a relative WER reduction of 51.5% in synthetic speech generated by neural TTS in the presentation domain. Further, we apply few-shot speaker adaptation for the E2E ASR by using a few utterances from target speakers in an unsupervised way, results in additional gains.