Improving the Inclusivity of Dutch Speech Recognition by Fine-tuning Whisper on the JASMIN-CGN Corpus
This work addresses inclusivity in speech recognition for specific Dutch subpopulations, but it is incremental as it applies an existing method to new data.
The paper tackled the problem of speech recognition inclusivity for underrepresented groups like children, the elderly, and non-native speakers in Dutch, by fine-tuning Whisper on the JASMIN-CGN corpus, resulting in relative WER reductions of up to 81% compared to zero-shot performance.
We test and study the variation in speech recognition of fine-tuned versions of the Whisper model on child, elderly and non-native Dutch speech from the JASMIN-CGN corpus. Our primary goal is to evaluate how speakers' age and linguistic background influence Whisper's performance. Whisper achieves varying Word Error Rates (WER) when fine-tuned on subpopulations of specific ages and linguistic backgrounds. Fine-tuned performance is remarkably better than zero-shot performance, achieving a relative reduction in WER of 81% for native children, 72% for non-native children, 67% for non-native adults, and 65% for native elderly people. Our findings underscore the importance of training speech recognition models like Whisper on underrepresented subpopulations such as children, the elderly, and non-native speakers.