Adaptation of Whisper models to child speech recognition
This work addresses the challenge of transcribing child speech for ASR applications, which is incremental as it adapts existing models rather than introducing a new paradigm.
The paper tackled the problem of automatic speech recognition (ASR) for child speech by adapting Whisper models through fine-tuning, resulting in significant improvements in ASR performance on child speech compared to non-finetuned models, though self-supervised wav2vec2 models fine-tuned on child speech outperformed Whisper fine-tuning.
Automatic Speech Recognition (ASR) systems often struggle with transcribing child speech due to the lack of large child speech datasets required to accurately train child-friendly ASR models. However, there are huge amounts of annotated adult speech datasets which were used to create multilingual ASR models, such as Whisper. Our work aims to explore whether such models can be adapted to child speech to improve ASR for children. In addition, we compare Whisper child-adaptations with finetuned self-supervised models, such as wav2vec2. We demonstrate that finetuning Whisper on child speech yields significant improvements in ASR performance on child speech, compared to non finetuned Whisper models. Additionally, utilizing self-supervised Wav2vec2 models that have been finetuned on child speech outperforms Whisper finetuning.