Segmentation-free Goodness of Pronunciation
This work addresses a bottleneck in computer-aided language learning systems by improving accuracy and flexibility for L2 learners, though it is incremental as it builds on existing GOP methods.
The paper tackles the problem of phoneme-level pronunciation assessment in mispronunciation detection and diagnosis by proposing segmentation-free methods, specifically GOP-SA and GOP-AF, which eliminate the need for pre-segmentation and enable the use of CTC-trained ASR models, achieving state-of-the-art results on the Speechocean762 dataset.
Mispronunciation detection and diagnosis (MDD) is a significant part in modern computer aided language learning (CALL) systems. Within MDD, phoneme-level pronunciation assessment is key to helping L2 learners improve their pronunciation. However, most systems are based on a form of goodness of pronunciation (GOP) which requires pre-segmentation of speech into phonetic units. This limits the accuracy of these methods and the possibility to use modern CTC-based acoustic models for their evaluation. In this study, we first propose self-alignment GOP (GOP-SA) that enables the use of CTC-trained ASR models for MDD. Next, we define a more general alignment-free method that takes all possible alignments of the target phoneme into account (GOP-AF). We give a theoretical account of our definition of GOP-AF, an implementation that solves potential numerical issues as well as a proper normalization which makes the method applicable with acoustic models with different peakiness over time. We provide extensive experimental results on the CMU Kids and Speechocean762 datasets comparing the different definitions of our methods, estimating the dependency of GOP-AF on the peakiness of the acoustic models and on the amount of context around the target phoneme. Finally, we compare our methods with recent studies over the Speechocean762 data showing that the feature vectors derived from the proposed method achieve state-of-the-art results on phoneme-level pronunciation assessment.