Fine-Tuning Whisper for Inclusive Prosodic Stress Analysis
This work addresses the challenge of prosodic stress analysis to improve equitable and robust transcription technologies for diverse populations, including neurodivergent individuals, though it is incremental as it builds on an existing model.
This study tackled the problem of analyzing prosodic stress in speech by fine-tuning OpenAI's Whisper large-v2 ASR model to recognize phrasal, lexical, and contrastive stress, achieving near-human accuracy in ASR performance across all stress types and near-perfect precision in classifying gender and neurotype from speech samples.
Prosody plays a crucial role in speech perception, influencing both human understanding and automatic speech recognition (ASR) systems. Despite its importance, prosodic stress remains under-studied due to the challenge of efficiently analyzing it. This study explores fine-tuning OpenAI's Whisper large-v2 ASR model to recognize phrasal, lexical, and contrastive stress in speech. Using a dataset of 66 native English speakers, including male, female, neurotypical, and neurodivergent individuals, we assess the model's ability to generalize stress patterns and classify speakers by neurotype and gender based on brief speech samples. Our results highlight near-human accuracy in ASR performance across all three stress types and near-perfect precision in classifying gender and neurotype. By improving prosody-aware ASR, this work contributes to equitable and robust transcription technologies for diverse populations.