Automatic Estimation of Intelligibility Measure for Consonants in Speech
This work addresses the problem of automating intelligibility assessment for speech segments, which is useful for researchers and clinicians in audiology and speech processing, but it is incremental as it builds on prior experiments and baseline methods.
The paper tackles the problem of automatically estimating the intelligibility of individual speech segments, specifically stop consonants, by training CNN regression models to predict the SNR at which humans can recognize consonants with 90% accuracy. The result is that the CNN models achieve less than 2 dB² mean squared error on average, outperforming a baseline ASR-based method which had a variance of 5.2 to 26.6 dB².
In this article, we provide a model to estimate a real-valued measure of the intelligibility of individual speech segments. We trained regression models based on Convolutional Neural Networks (CNN) for stop consonants \textipa{/p,t,k,b,d,g/} associated with vowel \textipa{/A/}, to estimate the corresponding Signal to Noise Ratio (SNR) at which the Consonant-Vowel (CV) sound becomes intelligible for Normal Hearing (NH) ears. The intelligibility measure for each sound is called SNR$_{90}$, and is defined to be the SNR level at which human participants are able to recognize the consonant at least 90\% correctly, on average, as determined in prior experiments with NH subjects. Performance of the CNN is compared to a baseline prediction based on automatic speech recognition (ASR), specifically, a constant offset subtracted from the SNR at which the ASR becomes capable of correctly labeling the consonant. Compared to baseline, our models were able to accurately estimate the SNR$_{90}$~intelligibility measure with less than 2 [dB$^2$] Mean Squared Error (MSE) on average, while the baseline ASR-defined measure computes SNR$_{90}$~with a variance of 5.2 to 26.6 [dB$^2$], depending on the consonant.