Pre-trained Sentence Embeddings for Implicit Discourse Relation Classification
This work addresses the challenge of understanding semantics in text coherence for NLP researchers, but it is incremental as it builds on existing embedding methods.
The paper tackled the problem of implicit discourse relation classification by exploring pre-trained sentence embeddings as base representations in a neural network, finding that combined models yielded significant performance improvements on two out of three evaluations.
Implicit discourse relations bind smaller linguistic units into coherent texts. Automatic sense prediction for implicit relations is hard, because it requires understanding the semantics of the linked arguments. Furthermore, annotated datasets contain relatively few labeled examples, due to the scale of the phenomenon: on average each discourse relation encompasses several dozen words. In this paper, we explore the utility of pre-trained sentence embeddings as base representations in a neural network for implicit discourse relation sense classification. We present a series of experiments using both supervised end-to-end trained models and pre-trained sentence encoding techniques - SkipThought, Sent2vec and Infersent. The pre-trained embeddings are competitive with the end-to-end model, and the approaches are complementary, with combined models yielding significant performance improvements on two of the three evaluations.