On the Impact of Speech Recognition Errors in Passage Retrieval for Spoken Question Answering
This addresses a critical bottleneck for speech-based QA systems by examining an unexplored component, though it appears incremental as it builds on existing datasets and methods.
The paper tackles the problem of speech recognition errors degrading passage retrieval performance in spoken question answering systems, finding that retrieval performance degrades with both synthetic and natural ASR noise, with natural noise causing further degradation.
Interacting with a speech interface to query a Question Answering (QA) system is becoming increasingly popular. Typically, QA systems rely on passage retrieval to select candidate contexts and reading comprehension to extract the final answer. While there has been some attention to improving the reading comprehension part of QA systems against errors that automatic speech recognition (ASR) models introduce, the passage retrieval part remains unexplored. However, such errors can affect the performance of passage retrieval, leading to inferior end-to-end performance. To address this gap, we augment two existing large-scale passage ranking and open domain QA datasets with synthetic ASR noise and study the robustness of lexical and dense retrievers against questions with ASR noise. Furthermore, we study the generalizability of data augmentation techniques across different domains; with each domain being a different language dialect or accent. Finally, we create a new dataset with questions voiced by human users and use their transcriptions to show that the retrieval performance can further degrade when dealing with natural ASR noise instead of synthetic ASR noise.