Memory Augmented Lookup Dictionary based Language Modeling for Automatic Speech Recognition
This work addresses the long-tail prediction problem in ASR, which is crucial for improving recognition accuracy of rare words, though it is incremental as it builds on existing Transformer LM methods.
The paper tackles the challenge of predicting infrequent (long-tail) tokens in automatic speech recognition by proposing a memory augmented lookup dictionary based Transformer language model, which significantly reduces word/character error rates and tail token error rates on Chinese and English datasets without affecting decoding efficiency.
Recent studies have shown that using an external Language Model (LM) benefits the end-to-end Automatic Speech Recognition (ASR). However, predicting tokens that appear less frequently in the training set is still quite challenging. The long-tail prediction problems have been widely studied in many applications, but only been addressed by a few studies for ASR and LMs. In this paper, we propose a new memory augmented lookup dictionary based Transformer architecture for LM. The newly introduced lookup dictionary incorporates rich contextual information in training set, which is vital to correctly predict long-tail tokens. With intensive experiments on Chinese and English data sets, our proposed method is proved to outperform the baseline Transformer LM by a great margin on both word/character error rate and tail tokens error rate. This is achieved without impact on the decoding efficiency. Overall, we demonstrate the effectiveness of our proposed method in boosting the ASR decoding performance, especially for long-tail tokens.