Compositional Generalization in Spoken Language Understanding
This addresses the issue of model failure in practical scenarios for SLU applications, representing an incremental advancement with specific improvements in compositional tasks.
The paper tackles the problem of poor compositional generalization in spoken language understanding models by analyzing spurious slot correlations and proposing a compositional model with specialized loss and training. It shows significant performance improvements, achieving up to 5% F1 score gains over state-of-the-art BERT models on benchmark and compositional splits of ATIS and SNIPS datasets.
State-of-the-art spoken language understanding (SLU) models have shown tremendous success in benchmark SLU datasets, yet they still fail in many practical scenario due to the lack of model compositionality when trained on limited training data. In this paper, we study two types of compositionality: (a) novel slot combination, and (b) length generalization. We first conduct in-depth analysis, and find that state-of-the-art SLU models often learn spurious slot correlations during training, which leads to poor performance in both compositional cases. To mitigate these limitations, we create the first compositional splits of benchmark SLU datasets and we propose the first compositional SLU model, including compositional loss and paired training that tackle each compositional case respectively. On both benchmark and compositional splits in ATIS and SNIPS, we show that our compositional SLU model significantly outperforms (up to $5\%$ F1 score) state-of-the-art BERT SLU model.