Shortcut-Stacked Sentence Encoders for Multi-Domain Inference
This work addresses the problem of improving sentence encoding for natural language inference tasks across multiple domains, representing an incremental advancement with strong specific gains.
The paper tackles multi-domain natural language inference by proposing a shortcut-stacked sentence encoder based on bidirectional LSTM-RNNs with fine-tuning, achieving top non-ensemble single-model results in the EMNLP RepEval 2017 Shared Task and a new state-of-the-art on the SNLI dataset.
We present a simple sequential sentence encoder for multi-domain natural language inference. Our encoder is based on stacked bidirectional LSTM-RNNs with shortcut connections and fine-tuning of word embeddings. The overall supervised model uses the above encoder to encode two input sentences into two vectors, and then uses a classifier over the vector combination to label the relationship between these two sentences as that of entailment, contradiction, or neural. Our Shortcut-Stacked sentence encoders achieve strong improvements over existing encoders on matched and mismatched multi-domain natural language inference (top non-ensemble single-model result in the EMNLP RepEval 2017 Shared Task (Nangia et al., 2017)). Moreover, they achieve the new state-of-the-art encoding result on the original SNLI dataset (Bowman et al., 2015).