Data Techniques For Online End-to-end Speech Recognition
This work addresses the data sparsity issue for practitioners needing to quickly deploy ASR systems in new domains, but it is incremental as it applies existing methods to this specific scenario.
The paper tackled the problem of building online end-to-end speech recognition systems with limited in-domain data by exploring simple techniques like data augmentation, domain adaptation, and knowledge distillation, resulting in improved ASR performance in the target domain.
Practitioners often need to build ASR systems for new use cases in a short amount of time, given limited in-domain data. While recently developed end-to-end methods largely simplify the modeling pipelines, they still suffer from the data sparsity issue. In this work, we explore a few simple-to-implement techniques for building online ASR systems in an end-to-end fashion, with a small amount of transcribed data in the target domain. These techniques include data augmentation in the target domain, domain adaptation using models previously trained on a large source domain, and knowledge distillation on non-transcribed target domain data, using an adapted bi-directional model as the teacher; they are applicable in real scenarios with different types of resources. Our experiments demonstrate that each technique is independently useful in the improvement of the online ASR performance in the target domain.