Corpus Synthesis for Zero-shot ASR domain Adaptation using Large Language Models
This addresses the domain adaptation challenge for ASR systems in scenarios with no available target data, though it is incremental as it builds on existing synthesis methods.
The paper tackles the problem of adapting ASR models to new domains without target-domain data by generating synthetic corpora using LLMs and speech synthesis, achieving a 28% relative WER improvement on unseen domains.
While Automatic Speech Recognition (ASR) systems are widely used in many real-world applications, they often do not generalize well to new domains and need to be finetuned on data from these domains. However, target-domain data usually are not readily available in many scenarios. In this paper, we propose a new strategy for adapting ASR models to new target domains without any text or speech from those domains. To accomplish this, we propose a novel data synthesis pipeline that uses a Large Language Model (LLM) to generate a target domain text corpus, and a state-of-the-art controllable speech synthesis model to generate the corresponding speech. We propose a simple yet effective in-context instruction finetuning strategy to increase the effectiveness of LLM in generating text corpora for new domains. Experiments on the SLURP dataset show that the proposed method achieves an average relative word error rate improvement of $28\%$ on unseen target domains without any performance drop in source domains.