ASR Error Detection via Audio-Transcript entailment
This addresses the issue of ASR errors propagating to downstream applications in critical domains such as healthcare, though it is incremental as it builds on existing error detection methods.
The paper tackles the problem of detecting errors in automatic speech recognition (ASR) systems, particularly in critical domains like healthcare, by proposing a novel end-to-end audio-transcript entailment approach, achieving classification error rates of 26.2% on all transcription errors and 23% on medical errors, with improvements of 12% and 15.4% over a strong baseline.
Despite improved performances of the latest Automatic Speech Recognition (ASR) systems, transcription errors are still unavoidable. These errors can have a considerable impact in critical domains such as healthcare, when used to help with clinical documentation. Therefore, detecting ASR errors is a critical first step in preventing further error propagation to downstream applications. To this end, we propose a novel end-to-end approach for ASR error detection using audio-transcript entailment. To the best of our knowledge, we are the first to frame this problem as an end-to-end entailment task between the audio segment and its corresponding transcript segment. Our intuition is that there should be a bidirectional entailment between audio and transcript when there is no recognition error and vice versa. The proposed model utilizes an acoustic encoder and a linguistic encoder to model the speech and transcript respectively. The encoded representations of both modalities are fused to predict the entailment. Since doctor-patient conversations are used in our experiments, a particular emphasis is placed on medical terms. Our proposed model achieves classification error rates (CER) of 26.2% on all transcription errors and 23% on medical errors specifically, leading to improvements upon a strong baseline by 12% and 15.4%, respectively.