Use of Speech Impairment Severity for Dysarthric Speech Recognition
This work addresses dysarthric speech recognition for individuals with speech impairments, offering incremental improvements by integrating severity information into existing ASR systems.
The paper tackled the challenge of speaker-level diversity in dysarthric speech recognition by incorporating speech impairment severity alongside speaker-identity, achieving statistically significant word error rate reductions up to 4.78% (14.03% relative) and setting a new lowest published WER of 17.82% on the UASpeech dataset.
A key challenge in dysarthric speech recognition is the speaker-level diversity attributed to both speaker-identity associated factors such as gender, and speech impairment severity. Most prior researches on addressing this issue focused on using speaker-identity only. To this end, this paper proposes a novel set of techniques to use both severity and speaker-identity in dysarthric speech recognition: a) multitask training incorporating severity prediction error; b) speaker-severity aware auxiliary feature adaptation; and c) structured LHUC transforms separately conditioned on speaker-identity and severity. Experiments conducted on UASpeech suggest incorporating additional speech impairment severity into state-of-the-art hybrid DNN, E2E Conformer and pre-trained Wav2vec 2.0 ASR systems produced statistically significant WER reductions up to 4.78% (14.03% relative). Using the best system the lowest published WER of 17.82% (51.25% on very low intelligibility) was obtained on UASpeech.