LearnerVoice: A Dataset of Non-Native English Learners' Spontaneous Speech
This addresses the challenge of ASR systems for second language learners, who produce ungrammatical and disfluent speech, by providing a tailored dataset and showing significant performance gains, though it is incremental as it builds on existing models.
The authors tackled the problem of poor Automatic Speech Recognition (ASR) performance on non-native English learners' spontaneous speech by releasing LearnerVoice, a 50.04-hour dataset with transcriptions, and fine-tuning whisper-small.en on it reduced the word error rate (WER) by 44.2% to 10.26%.
Prevalent ungrammatical expressions and disfluencies in spontaneous speech from second language (L2) learners pose unique challenges to Automatic Speech Recognition (ASR) systems. However, few datasets are tailored to L2 learner speech. We publicly release LearnerVoice, a dataset consisting of 50.04 hours of audio and transcriptions of L2 learners' spontaneous speech. Our linguistic analysis reveals that transcriptions in our dataset contain L2S (L2 learner's Spontaneous speech) features, consisting of ungrammatical expressions and disfluencies (e.g., filler words, word repetitions, self-repairs, false starts), significantly more than native speech datasets. Fine-tuning whisper-small.en with LearnerVoice achieves a WER of 10.26%, 44.2% lower than vanilla whisper-small.en. Furthermore, our qualitative analysis indicates that 54.2% of errors from the vanilla model on LearnerVoice are attributable to L2S features, with 48.1% of them being reduced in the fine-tuned model.