A Paradigm for Interpreting Metrics and Identifying Critical Errors in Automatic Speech Recognition
For researchers and practitioners in ASR, this work provides a method to make complex metrics interpretable like WER/CER, though it is an incremental methodological contribution.
The authors propose a paradigm that converts any metric into an interpretable error rate (minED) for automatic speech recognition, enabling analysis of error severity from a human perspective. The approach parallels transcription errors with human perception, addressing the interpretability issue of metric-based embeddings.
The most commonly used metrics for evaluating automatic speech transcriptions, namely Word Error Rate (WER) and Character Error Rate (CER), have been heavily criticized for their poor correlation to human perception and their inability to take into account linguistic and semantic information. While metric-based embeddings, seeking to approximate human perception, have been proposed, their scores remain difficult to interpret, unlike WER and CER. In this article, we overcome this problem by proposing a paradigm that consists in incorporating a chosen metric into it in order to obtain an equivalent of the error rate: a Minimum Edit Distance (minED). This approach parallels transcription errors with their human perception, also allowing an original study of the severity of these errors from a human perspective.