Sequence-level Confidence Classifier for ASR Utterance Accuracy and Application to Acoustic Models
This work addresses the need for interpretable and stable confidence scores in ASR systems, enabling more effective acoustic model adaptation with minimal data, though it is incremental in improving existing adaptation methods.
The paper tackles the problem of unreliable confidence scores in ASR systems by proposing a sequence-level confidence classifier that aligns closely with ASR accuracy and remains stable across updates, achieving word error rate reductions of up to 8.5% in supervised adaptation and 5.9% in semi-supervised adaptation on a Microsoft voice search task.
Scores from traditional confidence classifiers (CCs) in automatic speech recognition (ASR) systems lack universal interpretation and vary with updates to the underlying confidence or acoustic models (AMs). In this work, we build interpretable confidence scores with an objective to closely align with ASR accuracy. We propose a new sequence-level CC with a richer context providing CC scores highly correlated with ASR accuracy and scores stable across CC updates. Hence, expanding CC applications. Recently, AM customization has gained traction with the widespread use of unified models. Conventional adaptation strategies that customize AM expect well-matched data for the target domain with gold-standard transcriptions. We propose a cost-effective method of using CC scores to select an optimal adaptation data set, where we maximize ASR gains from minimal data. We study data in various confidence ranges and optimally choose data for AM adaptation with KL-Divergence regularization. On the Microsoft voice search task, data selection for supervised adaptation using the sequence-level confidence scores achieves word error rate reduction (WERR) of 8.5% for row-convolution LSTM (RC-LSTM) and 5.2% for latency-controlled bidirectional LSTM (LC-BLSTM). In the semi-supervised case, with ASR hypotheses as labels, our method provides WERR of 5.9% and 2.8% for RC-LSTM and LC-BLSTM, respectively.