Sentence-State LSTM for Text Representation
This work addresses inefficiencies in text representation for NLP tasks, offering an incremental improvement over existing LSTM models.
The authors tackled the limitations of sequential Bi-directional LSTMs by proposing a Sentence-State LSTM with parallel word states and recurrent steps for simultaneous local and global information exchange, achieving highly competitive performance on classification and sequence labeling benchmarks compared to stacked BiLSTMs with similar parameters.
Bi-directional LSTMs are a powerful tool for text representation. On the other hand, they have been shown to suffer various limitations due to their sequential nature. We investigate an alternative LSTM structure for encoding text, which consists of a parallel state for each word. Recurrent steps are used to perform local and global information exchange between words simultaneously, rather than incremental reading of a sequence of words. Results on various classification and sequence labelling benchmarks show that the proposed model has strong representation power, giving highly competitive performances compared to stacked BiLSTM models with similar parameter numbers.