An Effective Non-Autoregressive Model for Spoken Language Understanding
This work addresses faster inference for task-oriented dialogue systems, offering a significant speed-up with improved accuracy, though it is an incremental advancement in non-autoregressive methods for SLU.
The authors tackled the uncoordinated-slot problem in non-autoregressive spoken language understanding models, which reduces inference latency but suffers from performance issues, by proposing a Layered-Refine Transformer that improves overall accuracy by 1.5% and speeds up inference by over 10 times compared to state-of-the-art baselines.
Spoken Language Understanding (SLU), a core component of the task-oriented dialogue system, expects a shorter inference latency due to the impatience of humans. Non-autoregressive SLU models clearly increase the inference speed but suffer uncoordinated-slot problems caused by the lack of sequential dependency information among each slot chunk. To gap this shortcoming, in this paper, we propose a novel non-autoregressive SLU model named Layered-Refine Transformer, which contains a Slot Label Generation (SLG) task and a Layered Refine Mechanism (LRM). SLG is defined as generating the next slot label with the token sequence and generated slot labels. With SLG, the non-autoregressive model can efficiently obtain dependency information during training and spend no extra time in inference. LRM predicts the preliminary SLU results from Transformer's middle states and utilizes them to guide the final prediction. Experiments on two public datasets indicate that our model significantly improves SLU performance (1.5\% on Overall accuracy) while substantially speed up (more than 10 times) the inference process over the state-of-the-art baseline.