SynthASR: Unlocking Synthetic Data for Speech Recognition
This addresses the high cost and dependency on production data for ASR training in specific domains like healthcare, though it is incremental as it builds on existing synthetic data and continual learning methods.
The paper tackles the problem of training end-to-end automatic speech recognition (ASR) models for new applications where data is scarce, by using synthetic speech generated from text-to-speech engines. The result is a 65% relative improvement in recognition performance on a new medication name application without degrading existing general applications.
End-to-end (E2E) automatic speech recognition (ASR) models have recently demonstrated superior performance over the traditional hybrid ASR models. Training an E2E ASR model requires a large amount of data which is not only expensive but may also raise dependency on production data. At the same time, synthetic speech generated by the state-of-the-art text-to-speech (TTS) engines has advanced to near-human naturalness. In this work, we propose to utilize synthetic speech for ASR training (SynthASR) in applications where data is sparse or hard to get for ASR model training. In addition, we apply continual learning with a novel multi-stage training strategy to address catastrophic forgetting, achieved by a mix of weighted multi-style training, data augmentation, encoder freezing, and parameter regularization. In our experiments conducted on in-house datasets for a new application of recognizing medication names, training ASR RNN-T models with synthetic audio via the proposed multi-stage training improved the recognition performance on new application by more than 65% relative, without degradation on existing general applications. Our observations show that SynthASR holds great promise in training the state-of-the-art large-scale E2E ASR models for new applications while reducing the costs and dependency on production data.