Jointly Trained Sequential Labeling and Classification by Sparse Attention Neural Networks
This work addresses the challenge of leveraging correlations between classification and labeling tasks for researchers in natural language processing, but it appears incremental as it builds on existing LSTM-based approaches.
The paper tackles the problem of jointly modeling sentence-level classification and sequential labeling tasks, which are often correlated in language understanding, by proposing a jointly trained model using LSTM networks with a novel sparse attention mechanism. The result is that the method outperforms baseline models on ATIS and TREC datasets, though specific numbers are not provided in the abstract.
Sentence-level classification and sequential labeling are two fundamental tasks in language understanding. While these two tasks are usually modeled separately, in reality, they are often correlated, for example in intent classification and slot filling, or in topic classification and named-entity recognition. In order to utilize the potential benefits from their correlations, we propose a jointly trained model for learning the two tasks simultaneously via Long Short-Term Memory (LSTM) networks. This model predicts the sentence-level category and the word-level label sequence from the stepwise output hidden representations of LSTM. We also introduce a novel mechanism of "sparse attention" to weigh words differently based on their semantic relevance to sentence-level classification. The proposed method outperforms baseline models on ATIS and TREC datasets.