Utterance-level neural confidence measure for end-to-end children speech recognition
This work addresses the need for robust confidence measures in ASR systems for children's speech, where performance gaps exist, but it is incremental as it builds on existing E2E architectures and focuses on feature evaluation.
The study tackled the problem of utterance-level neural confidence measure for end-to-end children speech recognition, finding that acoustic-based features outperform linguistic ones and N-best scores are superior to single-best scores, with specific metrics like EER and AUC deemed inappropriate for mismatched ASR systems.
Confidence measure is a performance index of particular importance for automatic speech recognition (ASR) systems deployed in real-world scenarios. In the present study, utterance-level neural confidence measure (NCM) in end-to-end automatic speech recognition (E2E ASR) is investigated. The E2E system adopts the joint CTC-attention Transformer architecture. The prediction of NCM is formulated as a task of binary classification, i.e., accept/reject the input utterance, based on a set of predictor features acquired during the ASR decoding process. The investigation is focused on evaluating and comparing the efficacies of predictor features that are derived from different internal and external modules of the E2E system. Experiments are carried out on children speech, for which state-of-the-art ASR systems show less than satisfactory performance and robust confidence measure is particularly useful. It is noted that predictor features related to acoustic information of speech play a more important role in estimating confidence measure than those related to linguistic information. N-best score features show significantly better performance than single-best ones. It has also been shown that the metrics of EER and AUC are not appropriate to evaluate the NCM of a mismatched ASR with significant performance gap.