Training Autoregressive Speech Recognition Models with Limited in-domain Supervision
This addresses the challenge of limited supervision for conversational speech recognition, which is incremental as it builds on existing self-supervised methods.
The paper tackles the problem of training speech recognition models for conversational speech with limited in-domain transcribed data, by using self-supervised learning and pseudotranscription from a pre-trained model, resulting in an 8% absolute reduction in word error rate compared to a baseline.
Advances in self-supervised learning have significantly reduced the amount of transcribed audio required for training. However, the majority of work in this area is focused on read speech. We explore limited supervision in the domain of conversational speech. While we assume the amount of in-domain data is limited, we augment the model with open source read speech data. The XLS-R model has been shown to perform well with limited adaptation data and serves as a strong baseline. We use untranscribed data for self-supervised learning and semi-supervised training in an autoregressive encoder-decoder model. We demonstrate that by using the XLS-R model for pseudotranscription, a much smaller autoregressive model can outperform a finetuned XLS-R model when transcribed in-domain data is limited, reducing WER by as much as 8% absolute.