Improving Deliberation by Text-Only and Semi-Supervised Training
This work addresses speech recognition accuracy for applications like voice search, though it appears incremental as it builds on existing deliberation and training methods.
The paper tackled the problem of improving speech recognition accuracy by incorporating text-only and semi-supervised training into an attention-based deliberation model, achieving a 4%-12% word error rate (WER) reduction across tasks and an 11% relative WER reduction compared to a state-of-the-art language model rescoring method on Google Voice Search.
Text-only and semi-supervised training based on audio-only data has gained popularity recently due to the wide availability of unlabeled text and speech data. In this work, we propose incorporating text-only and semi-supervised training into an attention-based deliberation model. By incorporating text-only data in training a bidirectional encoder representation from transformer (BERT) for the deliberation text encoder, and large-scale text-to-speech and audio-only utterances using joint acoustic and text decoder (JATD) and semi-supervised training, we achieved 4%-12% WER reduction for various tasks compared to the baseline deliberation. Compared to a state-of-the-art language model (LM) rescoring method, the deliberation model reduces the Google Voice Search WER by 11% relative. We show that the deliberation model also achieves a positive human side-by-side evaluation compared to the state-of-the-art LM rescorer with reasonable endpointer latencies.