Building Robust Spoken Language Understanding by Cross Attention between Phoneme Sequence and ASR Hypothesis
This addresses robustness in voice-enabled virtual assistants, but it appears incremental as it builds on existing techniques for handling ASR errors.
The paper tackles the problem of making Spoken Language Understanding (SLU) robust to Automatic Speech Recognition (ASR) errors by proposing a model that uses cross attention between phoneme sequences and ASR hypotheses, showing effectiveness and competitiveness in experiments on three datasets.
Building Spoken Language Understanding (SLU) robust to Automatic Speech Recognition (ASR) errors is an essential issue for various voice-enabled virtual assistants. Considering that most ASR errors are caused by phonetic confusion between similar-sounding expressions, intuitively, leveraging the phoneme sequence of speech can complement ASR hypothesis and enhance the robustness of SLU. This paper proposes a novel model with Cross Attention for SLU (denoted as CASLU). The cross attention block is devised to catch the fine-grained interactions between phoneme and word embeddings in order to make the joint representations catch the phonetic and semantic features of input simultaneously and for overcoming the ASR errors in downstream natural language understanding (NLU) tasks. Extensive experiments are conducted on three datasets, showing the effectiveness and competitiveness of our approach. Additionally, We also validate the universality of CASLU and prove its complementarity when combining with other robust SLU techniques.