Goodness-of-pronunciation without phoneme time alignment
This work addresses the challenge of speech evaluation for low-resource languages, though it is incremental as it adapts existing weakly-supervised models rather than introducing a new paradigm.
The paper tackled the problem of expanding speech evaluation to low-resource languages by overcoming incompatibilities with weakly-supervised ASR models, achieving performance comparable to standard methods on English and Tamil datasets.
In speech evaluation, an Automatic Speech Recognition (ASR) model often computes time boundaries and phoneme posteriors for input features. However, limited data for ASR training hinders expansion of speech evaluation to low-resource languages. Open-source weakly-supervised models are capable of ASR over many languages, but they are frame-asynchronous and not phonemic, hindering feature extraction for speech evaluation. This paper proposes to overcome incompatibilities for feature extraction with weakly-supervised models, easing expansion of speech evaluation to low-resource languages. Phoneme posteriors are computed by mapping ASR hypotheses to a phoneme confusion network. Word instead of phoneme-level speaking rate and duration are used. Phoneme and frame-level features are combined using a cross-attention architecture, obviating phoneme time alignment. This performs comparably with standard frame-synchronous features on English speechocean762 and low-resource Tamil datasets.