OLMoASR: Open Models and Data for Training Robust Speech Recognition Models
This work addresses the need for open, high-quality data and models in speech recognition, enabling further research in robust speech processing, though it is incremental as it builds on existing methods like Whisper.
The paper tackles the problem of training robust zero-shot speech recognition models by introducing a large-scale dataset (OLMoASR-Pool with 3M hours) and a curated high-quality dataset (OLMoASR-Mix with 1M hours), resulting in models that achieve comparable performance to OpenAI's Whisper, with OLMoASR-medium.en attaining 12.8% and 11.0% WER on short and long-form benchmarks.
Improvements in training data scale and quality have led to significant advances, yet its influence in speech recognition remains underexplored. In this paper, we present a large-scale dataset, OLMoASR-Pool, and series of models, OLMoASR, to study and develop robust zero-shot speech recognition models. Beginning from OLMoASR-Pool, a collection of 3M hours of English audio and 17M transcripts, we design text heuristic filters to remove low-quality or mistranscribed data. Our curation pipeline produces a new dataset containing 1M hours of high-quality audio-transcript pairs, which we call OLMoASR-Mix. We use OLMoASR-Mix to train the OLMoASR-Mix suite of models, ranging from 39M (tiny.en) to 1.5B (large.en) parameters. Across all model scales, OLMoASR achieves comparable average performance to OpenAI's Whisper on short and long-form speech recognition benchmarks. Notably, OLMoASR-medium.en attains a 12.8\% and 11.0\% word error rate (WER) that is on par with Whisper's largest English-only model Whisper-medium.en's 12.4\% and 10.5\% WER for short and long-form recognition respectively (at equivalent parameter count). OLMoASR-Pool, OLMoASR models, and filtering, training and evaluation code will be made publicly available to further research on robust speech processing.