An Attention-over-Attention Generative Model for Joint Multiple Intent Detection and Slot Filling
This addresses the challenge of users expressing multiple intents in utterances for task-oriented dialogue systems, though it is incremental as it builds on existing methods.
The paper tackles the problem of multiple intent detection and slot filling in spoken language understanding by proposing an attention-over-attention generative model, achieving state-of-the-art performance on public and constructed datasets.
In task-oriented dialogue systems, spoken language understanding (SLU) is a critical component, which consists of two sub-tasks, intent detection and slot filling. Most existing methods focus on the single-intent SLU, where each utterance only has one intent. However, in real-world scenarios users usually express multiple intents in an utterance, which poses a challenge for existing dialogue systems and datasets. In this paper, we propose a generative framework to simultaneously address multiple intent detection and slot filling. In particular, an attention-over-attention decoder is proposed to handle the variable number of intents and the interference between the two sub-tasks by incorporating an inductive bias into the process of multi-task learning. Besides, we construct two new multi-intent SLU datasets based on single-intent utterances by taking advantage of the next sentence prediction (NSP) head of the BERT model. Experimental results demonstrate that our proposed attention-over-attention generative model achieves state-of-the-art performance on two public datasets, MixATIS and MixSNIPS, and our constructed datasets.