MultiPA: A Multi-task Speech Pronunciation Assessment Model for Open Response Scenarios
This work addresses the need for comprehensive pronunciation feedback in language learning applications, offering an incremental improvement over single-task models.
The authors tackled the problem of limited scope in open-response pronunciation assessment by developing MultiPA, a multi-task model that evaluates sentence-level accuracy, fluency, prosody, and word-level accuracy, achieving state-of-the-art performance on in-domain datasets and generalizing effectively to a new out-of-domain dataset.
Pronunciation assessment models designed for open response scenarios enable users to practice language skills in a manner similar to real-life communication. However, previous open-response pronunciation assessment models have predominantly focused on a single pronunciation task, such as sentence-level accuracy, rather than offering a comprehensive assessment in various aspects. We propose MultiPA, a Multitask Pronunciation Assessment model that provides sentence-level accuracy, fluency, prosody, and word-level accuracy assessment for open responses. We examined the correlation between different pronunciation tasks and showed the benefits of multi-task learning. Our model reached the state-of-the-art performance on existing in-domain data sets and effectively generalized to an out-of-domain dataset that we newly collected. The experimental results demonstrate the practical utility of our model in real-world applications.