Mispronunciation detection using self-supervised speech representations
This work addresses pronunciation assessment for language learners, but it is incremental as it applies existing SSL techniques to a specific domain.
The paper tackled mispronunciation detection for second language learners by comparing self-supervised learning (SSL) models with traditional methods, finding that training directly for the target task using non-native data yielded the best performance, with most SSL models performing similarly.
In recent years, self-supervised learning (SSL) models have produced promising results in a variety of speech-processing tasks, especially in contexts of data scarcity. In this paper, we study the use of SSL models for the task of mispronunciation detection for second language learners. We compare two downstream approaches: 1) training the model for phone recognition (PR) using native English data, and 2) training a model directly for the target task using non-native English data. We compare the performance of these two approaches for various SSL representations as well as a representation extracted from a traditional DNN-based speech recognition model. We evaluate the models on L2Arctic and EpaDB, two datasets of non-native speech annotated with pronunciation labels at the phone level. Overall, we find that using a downstream model trained for the target task gives the best performance and that most upstream models perform similarly for the task.