BERT for Joint Intent Classification and Slot Filling
This work addresses natural language understanding for tasks like virtual assistants, but it is incremental as it applies an existing method (BERT) to a specific domain.
The paper tackled the problem of joint intent classification and slot filling in natural language understanding by proposing a model based on BERT, achieving significant improvements in accuracy and F1 scores on public benchmark datasets compared to previous methods.
Intent classification and slot filling are two essential tasks for natural language understanding. They often suffer from small-scale human-labeled training data, resulting in poor generalization capability, especially for rare words. Recently a new language representation model, BERT (Bidirectional Encoder Representations from Transformers), facilitates pre-training deep bidirectional representations on large-scale unlabeled corpora, and has created state-of-the-art models for a wide variety of natural language processing tasks after simple fine-tuning. However, there has not been much effort on exploring BERT for natural language understanding. In this work, we propose a joint intent classification and slot filling model based on BERT. Experimental results demonstrate that our proposed model achieves significant improvement on intent classification accuracy, slot filling F1, and sentence-level semantic frame accuracy on several public benchmark datasets, compared to the attention-based recurrent neural network models and slot-gated models.