Improved Factorized Neural Transducer Model For text-only Domain Adaptation
This work addresses domain adaptation for ASR systems, which is incremental as it builds upon existing factorized neural transducer methods to enhance integration and performance.
The paper tackled the challenge of adapting end-to-end automatic speech recognition models to out-of-domain datasets using only text data, by proposing an improved factorized neural transducer model that achieved relative accuracy improvements of 1.2% to 2.8% on source domains and up to 30.2% WER reduction on out-of-domain datasets compared to baselines.
Adapting End-to-End ASR models to out-of-domain datasets with text data is challenging. Factorized neural Transducer (FNT) aims to address this issue by introducing a separate vocabulary decoder to predict the vocabulary. Nonetheless, this approach has limitations in fusing acoustic and language information seamlessly. Moreover, a degradation in word error rate (WER) on the general test sets was also observed, leading to doubts about its overall performance. In response to this challenge, we present the improved factorized neural Transducer (IFNT) model structure designed to comprehensively integrate acoustic and language information while enabling effective text adaptation. We assess the performance of our proposed method on English and Mandarin datasets. The results indicate that IFNT not only surpasses the neural Transducer and FNT in baseline performance in both scenarios but also exhibits superior adaptation ability compared to FNT. On source domains, IFNT demonstrated statistically significant accuracy improvements, achieving a relative enhancement of 1.2% to 2.8% in baseline accuracy compared to the neural Transducer. On out-of-domain datasets, IFNT shows relative WER(CER) improvements of up to 30.2% over the standard neural Transducer with shallow fusion, and relative WER(CER) reductions ranging from 1.1% to 2.8% on test sets compared to the FNT model.