Towards Real-time Mispronunciation Detection in Kids' Speech
This work addresses the need for rapid feedback in educational applications for kids, but it is incremental as it builds on existing teacher-student learning methods with specific alignment improvements.
The paper tackled the problem of enabling real-time mispronunciation detection in children's speech by addressing latency issues in state-of-the-art bi-directional recurrent networks, resulting in decreased latency and improved detection rates on the CSLU kids' corpus, with a trade-off between these goals.
Modern mispronunciation detection and diagnosis systems have seen significant gains in accuracy due to the introduction of deep learning. However, these systems have not been evaluated for the ability to be run in real-time, an important factor in applications that provide rapid feedback. In particular, the state-of-the-art uses bi-directional recurrent networks, where a uni-directional network may be more appropriate. Teacher-student learning is a natural approach to use to improve a uni-directional model, but when using a CTC objective, this is limited by poor alignment of outputs to evidence. We address this limitation by trying two loss terms for improving the alignments of our models. One loss is an "alignment loss" term that encourages outputs only when features do not resemble silence. The other loss term uses a uni-directional model as teacher model to align the bi-directional model. Our proposed model uses these aligned bi-directional models as teacher models. Experiments on the CSLU kids' corpus show that these changes decrease the latency of the outputs, and improve the detection rates, with a trade-off between these goals.