HeySQuAD: A Spoken Question Answering Dataset
This work addresses the problem of noisy spoken question understanding for digital assistants and real-world applications, presenting an incremental dataset and performance improvements.
The study tackled the challenge of evaluating spoken question answering (SQA) systems by introducing HeySQuAD, a large-scale dataset with 76k human-spoken and 97k machine-generated questions, and found that training with transcribed human-spoken and original SQuAD questions improved accuracy by 12.51% on human-spoken questions, with a further 2.03% gain from higher-quality transcriptions.
Spoken question answering (SQA) systems are critical for digital assistants and other real-world use cases, but evaluating their performance is a challenge due to the importance of human-spoken questions. This study presents a new large-scale community-shared SQA dataset called HeySQuAD, which includes 76k human-spoken questions, 97k machine-generated questions, and their corresponding textual answers from the SQuAD QA dataset. Our goal is to measure the ability of machines to accurately understand noisy spoken questions and provide reliable answers. Through extensive testing, we demonstrate that training with transcribed human-spoken and original SQuAD questions leads to a significant improvement (12.51%) in answering human-spoken questions compared to training with only the original SQuAD textual questions. Moreover, evaluating with a higher-quality transcription can lead to a further improvement of 2.03%. This research has significant implications for the development of SQA systems and their ability to meet the needs of users in real-world scenarios.