Variational Inference-Based Dropout in Recurrent Neural Networks for Slot Filling in Spoken Language Understanding
This work addresses slot filling for spoken language understanding, but it is incremental as it extends an existing method to more architectures.
The paper tackled slot filling in spoken language understanding by generalizing variational inference-based dropout to advanced RNN architectures like GRU and bi-directional LSTM/GRU, resulting in significant F-measure improvements over baseline systems on the ATIS dataset, with bi-directional LSTM/GRU achieving the best score.
This paper proposes to generalize the variational recurrent neural network (RNN) with variational inference (VI)-based dropout regularization employed for the long short-term memory (LSTM) cells to more advanced RNN architectures like gated recurrent unit (GRU) and bi-directional LSTM/GRU. The new variational RNNs are employed for slot filling, which is an intriguing but challenging task in spoken language understanding. The experiments on the ATIS dataset suggest that the variational RNNs with the VI-based dropout regularization can significantly improve the naive dropout regularization RNNs-based baseline systems in terms of F-measure. Particularly, the variational RNN with bi-directional LSTM/GRU obtains the best F-measure score.