Joint Automatic Speech Recognition And Structure Learning For Better Speech Understanding
This addresses the need for better spoken language understanding in speech applications, though it appears incremental as it builds on existing sequence-to-sequence methods.
The paper tackled the problem of simultaneous speech recognition and understanding by proposing a joint framework, achieving state-of-the-art performance on Chinese and English datasets with improved transcription and extraction capabilities.
Spoken language understanding (SLU) is a structure prediction task in the field of speech. Recently, many works on SLU that treat it as a sequence-to-sequence task have achieved great success. However, This method is not suitable for simultaneous speech recognition and understanding. In this paper, we propose a joint speech recognition and structure learning framework (JSRSL), an end-to-end SLU model based on span, which can accurately transcribe speech and extract structured content simultaneously. We conduct experiments on name entity recognition and intent classification using the Chinese dataset AISHELL-NER and the English dataset SLURP. The results show that our proposed method not only outperforms the traditional sequence-to-sequence method in both transcription and extraction capabilities but also achieves state-of-the-art performance on the two datasets.