Leveraging Sentence-level Information with Encoder LSTM for Semantic Slot Filling
This work addresses a key component in natural language understanding for applications like dialogue systems, but it is incremental as it builds on existing LSTM methods.
The paper tackled the problem of semantic slot filling by enhancing LSTM-based sequence labeling to model label dependencies and incorporate global sentence-level information, achieving a state-of-the-art F1-score of 95.66% on the ATIS corpus.
Recurrent Neural Network (RNN) and one of its specific architectures, Long Short-Term Memory (LSTM), have been widely used for sequence labeling. In this paper, we first enhance LSTM-based sequence labeling to explicitly model label dependencies. Then we propose another enhancement to incorporate the global information spanning over the whole input sequence. The latter proposed method, encoder-labeler LSTM, first encodes the whole input sequence into a fixed length vector with the encoder LSTM, and then uses this encoded vector as the initial state of another LSTM for sequence labeling. Combining these methods, we can predict the label sequence with considering label dependencies and information of whole input sequence. In the experiments of a slot filling task, which is an essential component of natural language understanding, with using the standard ATIS corpus, we achieved the state-of-the-art F1-score of 95.66%.