Phoneme-aware Encoding for Prefix-tree-based Contextual ASR
This work addresses the challenge of accurately recognizing rare words in speech recognition, particularly for proper nouns with atypical pronunciations, representing an incremental improvement to existing biasing methods.
The paper tackled the problem of recognizing context-specific rare words with unusual pronunciations in speech recognition by extending the Tree-constrained Pointer Generator (TCPGen) with phoneme-aware encoding, resulting in improved performance over grapheme-based encoding on English LibriSpeech and Japanese CSJ datasets.
In speech recognition applications, it is important to recognize context-specific rare words, such as proper nouns. Tree-constrained Pointer Generator (TCPGen) has shown promise for this purpose, which efficiently biases such words with a prefix tree. While the original TCPGen relies on grapheme-based encoding, we propose extending it with phoneme-aware encoding to better recognize words of unusual pronunciations. As TCPGen handles biasing words as subword units, we propose obtaining subword-level phoneme-aware encoding by using alignment between phonemes and subwords. Furthermore, we propose injecting phoneme-level predictions from CTC into queries of TCPGen so that the model better interprets the phoneme-aware encodings. We conducted ASR experiments with TCPGen for RNN transducer. We observed that proposed phoneme-aware encoding outperformed ordinary grapheme-based encoding on both the English LibriSpeech and Japanese CSJ datasets, demonstrating the robustness of our approach across linguistically diverse languages.