What Was That Again? Certified Robustness for Automatic Speech Recognition
For practitioners deploying ASR systems, this work provides a method to improve robustness and reliability without requiring ground truth labels.
The paper tackles the problem of detecting and mitigating adversarial and benign perturbations in deployed Automatic Speech Recognition systems without oracle knowledge of true transcription. Their certification-inspired pipeline achieves up to a 55% relative reduction in Word Error Rate across four architectures.
Automatic Speech Recognition systems are notoriously both sensitive to adversarial and benign perturbations. While this has been repeatedly demonstrated using reference datasets, detecting such behaviors in deployed systems is incredibly challenging, due to the absence of oracle knowledge of the true transcription. We demonstrate that employing a certification-inspired mechanism can significantly decrease WER, increase recall, and decrease the Spearman correlation between confidence and WER. We achieve this through a dual-gate diagnostic pipeline: a Two-Sided Atomic Audit that accumulates statistical wealth to certify both token existence and adversarial exclusion, and a Rank-Based Tournament that selects the winning sequence. Our evaluations across four diverse architectures demonstrate up to a 55% relative reduction in Word Error Rate, while also providing granular word- and sentence-level certifications to enhance acoustic security.