InterBiasing: Boost Unseen Word Recognition through Biasing Intermediate Predictions
This addresses the issue of recognizing unseen words in speech recognition, particularly for proper nouns or new terms, but is incremental as it builds on existing Self-conditioned CTC techniques.
The paper tackles the problem of end-to-end speech recognition models being biased towards training vocabulary, which leads to inaccurate recognition of unknown terms or proper nouns, and proposes an adaptation parameter-free method based on Self-conditioned CTC that improved the F1 score for unknown words in Japanese experiments.
Despite recent advances in end-to-end speech recognition methods, their output is biased to the training data's vocabulary, resulting in inaccurate recognition of unknown terms or proper nouns. To improve the recognition accuracy for a given set of such terms, we propose an adaptation parameter-free approach based on Self-conditioned CTC. Our method improves the recognition accuracy of misrecognized target keywords by substituting their intermediate CTC predictions with corrected labels, which are then passed on to the subsequent layers. First, we create pairs of correct labels and recognition error instances for a keyword list using Text-to-Speech and a recognition model. We use these pairs to replace intermediate prediction errors by the labels. Conditioning the subsequent layers of the encoder on the labels, it is possible to acoustically evaluate the target keywords. Experiments conducted in Japanese demonstrated that our method successfully improved the F1 score for unknown words.