Fast Word Error Rate Estimation Using Self-Supervised Representations for Speech and Text
This addresses the need for fast WER estimation in ASR systems, offering a practical solution for developers, but it is incremental as it builds on existing self-supervised methods.
The paper tackled the problem of efficiently estimating word error rate (WER) for automatic speech recognition without ground-truth labels by introducing Fe-WER, which uses self-supervised representations and achieved a 14.10% improvement in root mean square error and 3.4 times faster inference speed on Ted-Lium3.
Word error rate (WER) estimation aims to evaluate the quality of an automatic speech recognition (ASR) system's output without requiring ground-truth labels. This task has gained increasing attention as advanced ASR systems are trained on large amounts of data. In this context, the computational efficiency of a WER estimator becomes essential in practice. However, previous works have not prioritised this aspect. In this paper, a Fast estimator for WER (Fe-WER) is introduced, utilizing average pooling over self-supervised learning representations for speech and text. Our results demonstrate that Fe-WER outperformed a baseline relatively by 14.10% in root mean square error and 1.22% in Pearson correlation coefficient on Ted-Lium3. Moreover, a comparative analysis of the distributions of target WER and WER estimates was conducted, including an examination of the average values per speaker. Lastly, the inference speed was approximately 3.4 times faster in the real-time factor.